{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# XGBoost Parameter Tuning for Rent Listing Inqueries Dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Rental Listing Inquiries数据集是Kaggle平台上的一个分类竞赛任务，需要根据公寓的特征来预测其受欢迎程度（用户感兴趣程度分为高、中、低三类）。其中房屋的特征x共有14维，响应值y为用户对该公寓的感兴趣程度。评价标准为logloss"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1.参数调优：n_estimators"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "直接调用xgboost内嵌的cv寻找最佳的参数n_estimators"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from xgboost import XGBClassifier\n",
    "import xgboost as xgb\n",
    "\n",
    "import pandas as pd \n",
    "import numpy as np\n",
    "\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "from sklearn.model_selection import StratifiedKFold\n",
    "\n",
    "from sklearn.metrics import log_loss\n",
    "\n",
    "from matplotlib import pyplot\n",
    "import seaborn as sns\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 读取数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>bathrooms</th>\n",
       "      <th>bedrooms</th>\n",
       "      <th>price</th>\n",
       "      <th>price_bathrooms</th>\n",
       "      <th>price_bedrooms</th>\n",
       "      <th>room_diff</th>\n",
       "      <th>room_num</th>\n",
       "      <th>Year</th>\n",
       "      <th>Month</th>\n",
       "      <th>Day</th>\n",
       "      <th>...</th>\n",
       "      <th>walk</th>\n",
       "      <th>walls</th>\n",
       "      <th>war</th>\n",
       "      <th>washer</th>\n",
       "      <th>water</th>\n",
       "      <th>wheelchair</th>\n",
       "      <th>wifi</th>\n",
       "      <th>windows</th>\n",
       "      <th>work</th>\n",
       "      <th>interest_level</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.5</td>\n",
       "      <td>3</td>\n",
       "      <td>3000</td>\n",
       "      <td>1200.0</td>\n",
       "      <td>750.000000</td>\n",
       "      <td>-1.5</td>\n",
       "      <td>4.5</td>\n",
       "      <td>2016</td>\n",
       "      <td>6</td>\n",
       "      <td>24</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1.0</td>\n",
       "      <td>2</td>\n",
       "      <td>5465</td>\n",
       "      <td>2732.5</td>\n",
       "      <td>1821.666667</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2016</td>\n",
       "      <td>6</td>\n",
       "      <td>12</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>2850</td>\n",
       "      <td>1425.0</td>\n",
       "      <td>1425.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2016</td>\n",
       "      <td>4</td>\n",
       "      <td>17</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>3275</td>\n",
       "      <td>1637.5</td>\n",
       "      <td>1637.500000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2016</td>\n",
       "      <td>4</td>\n",
       "      <td>18</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1.0</td>\n",
       "      <td>4</td>\n",
       "      <td>3350</td>\n",
       "      <td>1675.0</td>\n",
       "      <td>670.000000</td>\n",
       "      <td>-3.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>2016</td>\n",
       "      <td>4</td>\n",
       "      <td>28</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 228 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   bathrooms  bedrooms  price  price_bathrooms  price_bedrooms  room_diff  \\\n",
       "0        1.5         3   3000           1200.0      750.000000       -1.5   \n",
       "1        1.0         2   5465           2732.5     1821.666667       -1.0   \n",
       "2        1.0         1   2850           1425.0     1425.000000        0.0   \n",
       "3        1.0         1   3275           1637.5     1637.500000        0.0   \n",
       "4        1.0         4   3350           1675.0      670.000000       -3.0   \n",
       "\n",
       "   room_num  Year  Month  Day       ...        walk  walls  war  washer  \\\n",
       "0       4.5  2016      6   24       ...           0      0    0       0   \n",
       "1       3.0  2016      6   12       ...           0      0    0       0   \n",
       "2       2.0  2016      4   17       ...           0      0    0       0   \n",
       "3       2.0  2016      4   18       ...           0      0    0       0   \n",
       "4       5.0  2016      4   28       ...           0      0    1       0   \n",
       "\n",
       "   water  wheelchair  wifi  windows  work  interest_level  \n",
       "0      0           0     0        0     0               1  \n",
       "1      0           0     0        0     0               2  \n",
       "2      0           0     0        0     0               0  \n",
       "3      0           0     0        0     0               2  \n",
       "4      0           0     0        0     0               2  \n",
       "\n",
       "[5 rows x 228 columns]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# path to where the data lies\n",
    "dpath = './data/'\n",
    "train = pd.read_csv(dpath +\"RentListingInquries_FE_train.csv\")\n",
    "test = pd.read_csv(dpath +\"RentListingInquries_FE_test.csv\")\n",
    "train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>bathrooms</th>\n",
       "      <th>bedrooms</th>\n",
       "      <th>price</th>\n",
       "      <th>price_bathrooms</th>\n",
       "      <th>price_bedrooms</th>\n",
       "      <th>room_diff</th>\n",
       "      <th>room_num</th>\n",
       "      <th>Year</th>\n",
       "      <th>Month</th>\n",
       "      <th>Day</th>\n",
       "      <th>...</th>\n",
       "      <th>virtual</th>\n",
       "      <th>walk</th>\n",
       "      <th>walls</th>\n",
       "      <th>war</th>\n",
       "      <th>washer</th>\n",
       "      <th>water</th>\n",
       "      <th>wheelchair</th>\n",
       "      <th>wifi</th>\n",
       "      <th>windows</th>\n",
       "      <th>work</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>2950</td>\n",
       "      <td>1475.000000</td>\n",
       "      <td>1475.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2016</td>\n",
       "      <td>6</td>\n",
       "      <td>11</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1.0</td>\n",
       "      <td>2</td>\n",
       "      <td>2850</td>\n",
       "      <td>1425.000000</td>\n",
       "      <td>950.000000</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2016</td>\n",
       "      <td>6</td>\n",
       "      <td>24</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>3758</td>\n",
       "      <td>1879.000000</td>\n",
       "      <td>1879.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2016</td>\n",
       "      <td>6</td>\n",
       "      <td>3</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1.0</td>\n",
       "      <td>2</td>\n",
       "      <td>3300</td>\n",
       "      <td>1650.000000</td>\n",
       "      <td>1100.000000</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2016</td>\n",
       "      <td>6</td>\n",
       "      <td>11</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2.0</td>\n",
       "      <td>2</td>\n",
       "      <td>4900</td>\n",
       "      <td>1633.333333</td>\n",
       "      <td>1633.333333</td>\n",
       "      <td>0.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>2016</td>\n",
       "      <td>4</td>\n",
       "      <td>12</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 227 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   bathrooms  bedrooms  price  price_bathrooms  price_bedrooms  room_diff  \\\n",
       "0        1.0         1   2950      1475.000000     1475.000000        0.0   \n",
       "1        1.0         2   2850      1425.000000      950.000000       -1.0   \n",
       "2        1.0         1   3758      1879.000000     1879.000000        0.0   \n",
       "3        1.0         2   3300      1650.000000     1100.000000       -1.0   \n",
       "4        2.0         2   4900      1633.333333     1633.333333        0.0   \n",
       "\n",
       "   room_num  Year  Month  Day  ...   virtual  walk  walls  war  washer  water  \\\n",
       "0       2.0  2016      6   11  ...         0     0      0    0       0      0   \n",
       "1       3.0  2016      6   24  ...         0     0      0    1       0      0   \n",
       "2       2.0  2016      6    3  ...         0     0      0    0       0      0   \n",
       "3       3.0  2016      6   11  ...         0     0      0    0       0      0   \n",
       "4       4.0  2016      4   12  ...         0     0      0    1       0      0   \n",
       "\n",
       "   wheelchair  wifi  windows  work  \n",
       "0           0     0        0     0  \n",
       "1           0     0        0     0  \n",
       "2           0     0        0     0  \n",
       "3           1     0        0     0  \n",
       "4           0     0        0     0  \n",
       "\n",
       "[5 rows x 227 columns]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "发现测试集中没有y，这次作业无法得到在测试集上的表现情况"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "*********** train **********\n",
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 49352 entries, 0 to 49351\n",
      "Columns: 228 entries, bathrooms to interest_level\n",
      "dtypes: float64(9), int64(219)\n",
      "memory usage: 85.8 MB\n",
      "************ test ***********\n",
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 74659 entries, 0 to 74658\n",
      "Columns: 227 entries, bathrooms to work\n",
      "dtypes: float64(9), int64(218)\n",
      "memory usage: 129.3 MB\n"
     ]
    }
   ],
   "source": [
    "print(\"*********** train **********\")\n",
    "train.info()\n",
    "print(\"************ test ***********\")\n",
    "test.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "测试集中少一维，即y标签"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZUAAAELCAYAAAARNxsIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAGx5JREFUeJzt3X2wHXWd5/H3h/AgPibIhcokYYKaGUUdo14hypSj6EDAHRMpUChHIkNtxAKF0rUEd5aMPMyM4ygrM8pupogkrkOIiBI1GDMMyII8JGB4CJHlioxcyZJgeNQVivDZP/p35eTm5N7OpU9OTu7nVdV1ur/96z7fwy340v379a9lm4iIiCbs0e0EIiJi95GiEhERjUlRiYiIxqSoREREY1JUIiKiMSkqERHRmBSViIhoTIpKREQ0JkUlIiIas2e3E9jZ9t9/f0+fPr3baURE9JTbbrvtEdt9o7Ubd0Vl+vTprFmzpttpRET0FEn/Uaddbn9FRERjOlZUJL1I0q2S7pC0TtLnS/xSSb+QtLYsM0tcki6SNCDpTklvaTnXPEn3lWVeS/ytku4qx1wkSZ36PRERMbpO3v56GjjC9lOS9gJukHR12fcZ21cMa380MKMshwEXA4dJ2g9YAPQDBm6TtNz2o6XNfOBmYAUwG7iaiIjoio5dqbjyVNncqywjzbM/B1hSjrsZmChpMnAUsMr25lJIVgGzy76X277J1fz9S4C5nfo9ERExuo72qUiaIGktsJGqMNxSdl1QbnFdKGmfEpsCPNhy+GCJjRQfbBNvl8d8SWskrdm0adML/l0REdFeR4uK7S22ZwJTgUMlvQE4G3gt8DZgP+CzpXm7/hCPId4uj4W2+2339/WNOiIuIiLGaKeM/rL9GHAdMNv2hnKL62ng68ChpdkgMK3lsKnAQ6PEp7aJR0REl3Ry9FefpIllfV/gvcDPSl8IZaTWXODucshy4KQyCmwW8LjtDcBK4EhJkyRNAo4EVpZ9T0qaVc51EnBVp35PRESMrpOjvyYDiyVNoCpey2x/X9K/S+qjun21Fji1tF8BHAMMAL8FTgawvVnSecDq0u5c25vL+seBS4F9qUZ9ZeRXREQXqRo4NX709/c7T9RH7NoO/6fDu53Cbu/GT9y4Q+0l3Wa7f7R2eaI+IiIak6ISERGNSVGJiIjGpKhERERjUlQiIqIxKSoREdGYFJWIiGhMikpERDQmRSUiIhqTohIREY1JUYmIiMakqERERGNSVCIiojEpKhER0ZgUlYiIaEyKSkRENCZFJSIiGpOiEhERjUlRiYiIxqSoREREY1JUIiKiMR0rKpJeJOlWSXdIWifp8yV+sKRbJN0n6XJJe5f4PmV7oOyf3nKus0v8XklHtcRnl9iApLM69VsiIqKeTl6pPA0cYftNwExgtqRZwBeAC23PAB4FTintTwEetf0a4MLSDkmHACcArwdmA1+TNEHSBOCrwNHAIcCJpW1ERHRJx4qKK0+Vzb3KYuAI4IoSXwzMLetzyjZl/3skqcSX2n7a9i+AAeDQsgzYvt/2M8DS0jYiIrqko30q5YpiLbARWAX8HHjM9rOlySAwpaxPAR4EKPsfB17ZGh92zPbiERHRJR0tKra32J4JTKW6snhdu2blU9vZt6PxbUiaL2mNpDWbNm0aPfGIiBiTnTL6y/ZjwHXALGCipD3LrqnAQ2V9EJgGUPa/AtjcGh92zPbi7b5/oe1+2/19fX1N/KSIiGijk6O/+iRNLOv7Au8F1gPXAseVZvOAq8r68rJN2f/vtl3iJ5TRYQcDM4BbgdXAjDKabG+qzvzlnfo9ERExuj1HbzJmk4HFZZTWHsAy29+XdA+wVNL5wE+BS0r7S4BvSBqgukI5AcD2OknLgHuAZ4HTbG8BkHQ6sBKYACyyva6DvyciIkbRsaJi+07gzW3i91P1rwyP/w44fjvnugC4oE18BbDiBScbERGNyBP1ERHRmBSViIhoTIpKREQ0JkUlIiIak6ISERGNSVGJiIjGpKhERERjUlQiIqIxKSoREdGYFJWIiGjMqEVF0ksk7VHW/0jS+yXt1fnUIiKi19S5UrkeeJGkKcA1wMnApZ1MKiIielOdoiLbvwWOBf7J9geo3gkfERGxlVpFRdLbgQ8DPyixTk6ZHxERPapOUTkTOBv4Tnm3yauoXrQVERGxlVGvOGz/GPixpJeU7fuBT3Y6sYiI6D11Rn+9vbytcX3ZfpOkr3U8s4iI6Dl1bn/9d+Ao4NcAtu8A3tnJpCIiojfVevjR9oPDQls6kEtERPS4OqO4HpT0DsCS9qbqT1nf2bQiIqIX1blSORU4DZgCDAIzy3ZERMRWRi0qth+x/WHbB9o+wPZf2v71aMdJmibpWknrJa2TdEaJ/42kX0laW5ZjWo45W9KApHslHdUSn11iA5LOaokfLOkWSfdJurxcSUVERJfUGf21WNLElu1JkhbVOPezwKdtvw6YBZwmaehJ/AttzyzLinLeQ4ATgNcDs4GvSZogaQLwVeBoqif5T2w5zxfKuWYAjwKn1MgrIiI6pM7trz+x/djQhu1HgTePdpDtDbZvL+tPUvXDTBnhkDnAUttP2/4FMAAcWpYB2/fbfgZYCsyRJOAI4Ipy/GJgbo3fExERHVKnqOwhadLQhqT92MFpWiRNpypEt5TQ6ZLulLSo5dxTgNZRZoMltr34K4HHbD87LB4REV1Sp6h8CfiJpPMknQf8BPiHul8g6aXAt4EzbT8BXAy8mqrDf0M5P4DaHO4xxNvlMF/SGklrNm3aVDf1iIjYQXU66pcAxwEPAxuBY21/o87Jy3tXvg180/aV5XwP295i+zngX6hub0F1pTGt5fCpwEMjxB8BJkrac1i83W9YaLvfdn9fX1+d1CMiYgzqvvnxZ8CVwFXAU5IOGu2A0udxCbDe9pdb4pNbmn0AuLusLwdOkLSPpIOBGcCtwGpgRhnptTdVZ/5y26aa2PK4cvy8kl9ERHTJqH0jkj4BLKC6UtlCddvJwJ+McujhwEeAuyStLbHPUY3emlnO8QDwMYAyA/Iy4B6qkWOn2d5ScjgdWAlMABbZXlfO91lgqaTzgZ9SFbGIiOiSOh3uZwB/XOfZlFa2b6B9v8eKEY65ALigTXxFu+PKjMmHDo9HRER31Ln99SDweKcTiYiI3lfnSuV+4DpJPwCeHgq29pNERERAvaLyy7LsXZaIiIi26rz58fMAkl5i+zedTykiInpV3vwYERGNyZsfIyKiMXnzY0RENCZvfoyIiMbkzY8REdGYEa9UyguyPmL7wzspn4iI6GEjXqmUubfm7KRcIiKix9XpU7lR0j8DlwO/f05l6K2OERERQ+oUlXeUz3NbYqZ6lW9ERMTvjdansgdwse1lOymfiIjoYaP1qTwHnL6TcomIiB5XZ0jxKkn/RdI0SfsNLR3PLCIiek6dPpW/Kp+tz6YYeFXz6URERC+rM0vxwTsjkYiI6H113lF/Uru47SXNpxMREb2szu2vt7Wsvwh4D3A7kKISERFbqXP76xOt25JeAXyjYxlFRETPqjX1/TC/BWaM1qiMFrtW0npJ6ySdUeL7SVol6b7yOanEJekiSQOS7pT0lpZzzSvt75M0ryX+Vkl3lWMukqQx/J6IiGhInTc/fk/S8rJ8H7gXuKrGuZ8FPm37dcAs4DRJhwBnAdfYngFcU7YBjqYqVjOA+cDF5fv3AxYAhwGHAguGClFpM7/luNk18oqIiA6p06fyjy3rzwL/YXtwtINsbwA2lPUnJa2nmj5/DvCu0mwxcB3w2RJfYtvAzZImSppc2q6yvRlA0ipgtqTrgJfbvqnElwBzgatr/KaIiOiAOkXll8AG278DkLSvpOm2H6j7JZKmA28GbgEOLAUH2xskHVCaTQFa3zA5WGIjxQfbxCMiokvq9Kl8C3iuZXtLidUi6aXAt4EzbT8xUtM2MY8h3i6H+ZLWSFqzadOm0VKOiIgxqlNU9rT9zNBGWd+7zskl7UVVUL5p+8oSfrjc1qJ8bizxQWBay+FTgYdGiU9tE9+G7YW2+2339/X11Uk9IiLGoE5R2STp/UMbkuYAj4x2UBmJdQmw3vaXW3YtB4ZGcM3j+U7/5cBJZRTYLODxcptsJXCkpEmlg/5IYGXZ96SkWeW7TqLeAIKIiOiQOn0qpwLfLC/qguoKoe1T9sMcDnwEuEvS2hL7HPD3wDJJp1D11xxf9q0AjgEGqIYtnwxge7Ok84DVpd25Q532wMeBS4F9qTro00kfEdFFdR5+/Dkwq/SNyPaTdU5s+wba93tA9VT+8PZm60krW/ctAha1ia8B3lAnn4iI6Lw6z6n8raSJtp8qQ4MnSTp/ZyQXERG9pU6fytG2HxvasP0o1W2qiIiIrdQpKhMk7TO0IWlfYJ8R2kdExDhVp6P+fwHXSPo61XMgf0X1JHxERMRW6nTU/4OkO4H3ltB5tld2Nq2IiOhFda5UAH4K7EV1pfLTzqUTERG9rM7orw8CtwLHAR8EbpF0XKcTi4iI3lPnSuW/Am+zvRFAUh/wb8AVnUwsIiJ6T53RX3sMFZTi1zWPi4iIcabOlcoPJa0ELivbH6KaUiUiImIrdUZ/fUbSscCfUk27stD2dzqeWURE9Jxao7/KtPVXjtowIiLGtfSNREREY1JUIiKiMdstKpKuKZ9f2HnpRERELxupT2WypD8D3i9pKcPejWL79o5mFhERPWekonIOcBbVu9+/PGyfgSM6lVRERPSm7RYV21cAV0j6b7bP24k5RUREj6rznMp5kt4PvLOErrP9/c6mFRERvajOhJJ/B5wB3FOWM0osIiJiK3UefnwfMNP2cwCSFlNNf392JxOLiIjeU/c5lYkt66/oRCIREdH76hSVvwN+KunScpVyG/C3ox0kaZGkjZLubon9jaRfSVpblmNa9p0taUDSvZKOaonPLrEBSWe1xA+WdIuk+yRdLmnvuj86IiI6Y9SiYvsyYBbV3F9XAm+3vbTGuS8FZreJX2h7ZllWAEg6BDgBeH055muSJkiaAHwVOBo4BDixtAX4QjnXDOBR4JQaOUVERAfVuv1le4Pt5bavsv1/ax5zPbC5Zh5zgKW2n7b9C2AAOLQsA7bvt/0MsBSYI0lUz8kMvShsMTC35ndFRESHdGPur9Ml3Vluj00qsSnAgy1tBktse/FXAo/ZfnZYvC1J8yWtkbRm06ZNTf2OiIgYZmcXlYuBVwMzgQ3Al0pcbdp6DPG2bC+03W+7v6+vb8cyjoiI2kYsKpL2aO1of6FsP2x7Sxme/C9Ut7egutKY1tJ0KvDQCPFHgImS9hwWj4iILhqxqJT/+N8h6aAmvkzS5JbNDwBDBWs5cIKkfSQdDMwAbgVWAzPKSK+9qTrzl9s2cC1wXDl+HnBVEzlGRMTY1Xn4cTKwTtKtwG+GgrbfP9JBki4D3gXsL2kQWAC8S9JMqltVDwAfK+daJ2kZ1RP7zwKn2d5SznM6sBKYACyyva58xWeBpZLOp3oY85I6PzgiIjqnTlH5/FhObPvENuHt/off9gXABW3iK4AVbeL38/zts4iI2AXUmVDyx5L+EJhh+98kvZjqqiEiImIrdSaU/M9Uz4P8zxKaAny3k0lFRERvqjOk+DTgcOAJANv3AQd0MqmIiOhNdYrK0+VpdgDKMN7tPhMSERHjV52i8mNJnwP2lfTnwLeA73U2rYiI6EV1ispZwCbgLqohwCuAv+5kUhER0ZvqjP56rkx5fwvVba97y8OHERERWxm1qEh6H/A/gJ9Tzbl1sKSP2b6608lFRERvqfPw45eAd9seAJD0auAHQIpKRERspU6fysahglLcD2zsUD4REdHDtnulIunYsrpO0gpgGVWfyvFUEz1GRERsZaTbX3/Rsv4w8GdlfRMwadvmEREx3m23qNg+eWcmEhERva/O6K+DgU8A01vbjzb1fUREjD91Rn99l2rK+u8Bz3U2nYiI6GV1isrvbF/U8UwiIqLn1SkqX5G0APgR8PRQ0PbtHcsqIiJ6Up2i8kbgI8ARPH/7y2U7Ypf0y3Pf2O0UxoWDzrmr2ynELqZOUfkA8KrW6e8jIiLaqfNE/R3AxE4nEhERva/OlcqBwM8krWbrPpUMKY6IiK3UKSoLxnJiSYuA/0Q1d9gbSmw/4HKqZ14eAD5o+1FJAr4CHAP8Fvjo0EAASfN4/v0t59teXOJvBS4F9qV6x8sZmZI/IqK7Rr39ZfvH7ZYa574UmD0sdhZwje0ZwDVlG+BoYEZZ5gMXw++L0ALgMOBQYIGkoSliLi5th44b/l0REbGTjVpUJD0p6Ymy/E7SFklPjHac7euBzcPCc4DFZX0xMLclvsSVm4GJkiYDRwGrbG+2/SiwCphd9r3c9k3l6mRJy7kiIqJL6rz58WWt25LmUl01jMWBtjeU826QdECJTwEebGk3WGIjxQfbxNuSNJ/qqoaDDjpojKlHRMRo6oz+2ort79L8Mypq91VjiLdle6Htftv9fX19Y0wxIiJGU2dCyWNbNvcA+hnhP+CjeFjS5HKVMpnnX/Y1CExraTcVeKjE3zUsfl2JT23TPiIiuqjOlcpftCxHAU9S9YGMxXJgXlmfB1zVEj9JlVnA4+U22UrgSEmTSgf9kcDKsu9JSbPKyLGTWs4VERFdUqdPZUzvVZF0GdVVxv6SBqlGcf09sEzSKcAvqd4iCdWQ4GOAAaohxSeX794s6Tyef9PkubaHOv8/zvNDiq8uS0REdNFIrxM+Z4TjbPu8kU5s+8Tt7HpPu5MBp23nPIuARW3ia4A3jJRDRETsXCNdqfymTewlwCnAK4ERi0pERIw/I71O+EtD65JeBpxBdVtqKfCl7R0XERHj14h9KuWJ9k8BH6Z6WPEt5SHEiIiIbYzUp/JF4FhgIfBG20/ttKwiIqInjTSk+NPAH1BN5vhQy1QtT9aZpiUiIsafkfpUdvhp+4iIGN9SOCIiojEpKhER0ZgUlYiIaEyKSkRENCZFJSIiGpOiEhERjUlRiYiIxqSoREREY1JUIiKiMSkqERHRmBSViIhoTIpKREQ0JkUlIiIak6ISERGNSVGJiIjGdKWoSHpA0l2S1kpaU2L7SVol6b7yOanEJekiSQOS7pT0lpbzzCvt75M0rxu/JSIintfNK5V3255pu79snwVcY3sGcE3ZBjgamFGW+cDFUBUhYAFwGHAosGCoEEVERHfsSre/5gCLy/piYG5LfIkrNwMTJU0GjgJW2d5s+1FgFTB7ZycdERHP61ZRMfAjSbdJml9iB9reAFA+DyjxKcCDLccOltj24hER0SXbfUd9hx1u+yFJBwCrJP1shLZqE/MI8W1PUBWu+QAHHXTQjuYaERE1deVKxfZD5XMj8B2qPpGHy20tyufG0nwQmNZy+FTgoRHi7b5voe1+2/19fX1N/pSIiGix04uKpJdIetnQOnAkcDewHBgawTUPuKqsLwdOKqPAZgGPl9tjK4EjJU0qHfRHllhERHRJN25/HQh8R9LQ9/+r7R9KWg0sk3QK8Evg+NJ+BXAMMAD8FjgZwPZmSecBq0u7c21v3nk/IyIihtvpRcX2/cCb2sR/DbynTdzAads51yJgUdM5RkTE2OxKQ4ojIqLHpahERERjujWkuCe89TNLup3Cbu+2L57U7RQiokG5UomIiMakqERERGNSVCIiojEpKhER0ZgUlYiIaEyKSkRENCZFJSIiGpOiEhERjUlRiYiIxqSoREREY1JUIiKiMSkqERHRmBSViIhoTIpKREQ0JkUlIiIak6ISERGNSVGJiIjGpKhERERjer6oSJot6V5JA5LO6nY+ERHjWU8XFUkTgK8CRwOHACdKOqS7WUVEjF89XVSAQ4EB2/fbfgZYCszpck4REeNWrxeVKcCDLduDJRYREV2wZ7cTeIHUJuZtGknzgfll8ylJ93Y0q+7aH3ik20nUpX+c1+0UdiU99bcDYEG7fwXHrZ76++mTO/y3+8M6jXq9qAwC01q2pwIPDW9keyGwcGcl1U2S1tju73YesePyt+tt+ftVev3212pghqSDJe0NnAAs73JOERHjVk9fqdh+VtLpwEpgArDI9roupxURMW71dFEBsL0CWNHtPHYh4+I2324qf7velr8fIHubfu2IiIgx6fU+lYiI2IWkqOwmMl1N75K0SNJGSXd3O5fYMZKmSbpW0npJ6ySd0e2cui23v3YDZbqa/wP8OdUw69XAibbv6WpiUYukdwJPAUtsv6Hb+UR9kiYDk23fLullwG3A3PH8716uVHYPma6mh9m+Htjc7Txix9neYPv2sv4ksJ5xPqtHisruIdPVRHSZpOnAm4FbuptJd6Wo7B5qTVcTEZ0h6aXAt4EzbT/R7Xy6KUVl91BrupqIaJ6kvagKyjdtX9ntfLotRWX3kOlqIrpAkoBLgPW2v9ztfHYFKSq7AdvPAkPT1awHlmW6mt4h6TLgJuCPJQ1KOqXbOUVthwMfAY6QtLYsx3Q7qW7KkOKIiGhMrlQiIqIxKSoREdGYFJWIiGhMikpERDQmRSUiIhqTohIREY1JUYkAJP2kRpszJb24w3nMHO05B0kflfTPDX9v4+eM8SlFJQKw/Y4azc4EdqiolNcS7IiZwLh+eC56W4pKBCDpqfL5LknXSbpC0s8kfVOVTwJ/AFwr6drS9khJN0m6XdK3yqSCSHpA0jmSbgCOl/RqST+UdJuk/y3ptaXd8ZLulnSHpOvLFDvnAh8qT2Z/qEbefZK+LWl1WQ6XtEfJYWJLuwFJB7Zr3/g/zBjX9ux2AhG7oDcDr6ealPNG4HDbF0n6FPBu249I2h/4a+C9tn8j6bPAp6iKAsDvbP8pgKRrgFNt3yfpMOBrwBHAOcBRtn8laaLtZySdA/TbPr1mrl8BLrR9g6SDgJW2XyfpKuADwNfLdz5g+2FJ/zq8PfC6F/jPK+L3UlQitnWr7UEASWuB6cANw9rMAg4BbqzmFGRvqvm7hlxejn8p8A7gW6UdwD7l80bgUknLgLHObvte4JCWc7+8vIHwcqqi9XWqCUYvH6V9RCNSVCK29XTL+hba/3siYJXtE7dzjt+Uzz2Ax2zPHN7A9qnlKuJ9wFpJ27SpYQ/g7bb/31bJSTcBr5HUB8wFzh+l/Ri+OmJb6VOJqO9JYOj/6m8GDpf0GgBJL5b0R8MPKC9s+oWk40s7SXpTWX+17VtsnwM8QvVOnNbvqONHVDNUU845s3yvge8AX6aalv3XI7WPaEqKSkR9C4GrJV1rexPwUeAySXdSFZnXbue4DwOnSLoDWAfMKfEvSrpL0t3A9cAdwLVUt6dqddQDnwT6Jd0p6R7g1JZ9lwN/yfO3vkZrH/GCZer7iIhoTK5UIiKiMemoj9hFSToZOGNY+Ebbp3Ujn4g6cvsrIiIak9tfERHRmBSViIhoTIpKREQ0JkUlIiIak6ISERGN+f+UQGoL+CmGNwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.countplot(train.interest_level);\n",
    "pyplot.xlabel('interest_level');\n",
    "pyplot.ylabel('Number of occurrences');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "总共有三类样本，每类样本分布的并不均匀，一般类别之间的数量差别在十倍以上时才考虑使用class_weight，这里暂不考虑"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# drop ids and get labels\n",
    "y_train = train['interest_level']\n",
    "\n",
    "train = train.drop([\"interest_level\"], axis=1)\n",
    "X_train = np.array(train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# prepare cross validation\n",
    "kfold = StratifiedKFold(n_splits=5, shuffle=True, random_state=3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "默认参数，此时学习率为0.1，比较大，观察弱分类数目的大致范围 （采用默认参数配置，看看模型是过拟合还是欠拟合）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "#直接调用xgboost内嵌的交叉验证（cv），可对连续的n_estimators参数进行快速交叉验证\n",
    "#而GridSearchCV只能对有限个参数进行交叉验证\n",
    "def modelfit(alg, X_train, y_train, cv_folds=None, early_stopping_rounds=10):\n",
    "    xgb_param = alg.get_xgb_params()\n",
    "    xgb_param['num_class'] = 3\n",
    "    \n",
    "    #直接调用xgboost，而非sklarn的wrapper类\n",
    "    xgtrain = xgb.DMatrix(X_train, label = y_train)\n",
    "        \n",
    "    cvresult = xgb.cv(xgb_param, xgtrain, num_boost_round=alg.get_params()['n_estimators'], folds =cv_folds,\n",
    "             metrics='mlogloss', early_stopping_rounds=early_stopping_rounds)\n",
    "  \n",
    "    cvresult.to_csv('1_nestimators.csv', index_label = 'n_estimators')\n",
    "    \n",
    "    #最佳参数n_estimators\n",
    "    n_estimators = cvresult.shape[0]\n",
    "    \n",
    "    # 采用交叉验证得到的最佳参数n_estimators，训练模型\n",
    "    alg.set_params(n_estimators = n_estimators)\n",
    "    alg.fit(X_train, y_train, eval_metric='mlogloss')\n",
    "    \n",
    "    #Predict training set:\n",
    "    train_predprob = alg.predict_proba(X_train)\n",
    "    logloss = log_loss(y_train, train_predprob)\n",
    "        \n",
    "    #Print model report:\n",
    "    print (\"logloss of train :\" )\n",
    "    print (logloss)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "logloss of train :\n",
      "0.4884125621349346\n"
     ]
    }
   ],
   "source": [
    "#params = {\"objective\": \"multi:softprob\", \"eval_metric\":\"mlogloss\", \"num_class\": 9}\n",
    "xgb1 = XGBClassifier(\n",
    "        learning_rate =0.1,\n",
    "        n_estimators=1000,  #数值大没关系，cv会自动返回合适的n_estimators\n",
    "        max_depth=5,\n",
    "        min_child_weight=1,\n",
    "        gamma=0,\n",
    "        subsample=0.5,\n",
    "        colsample_bytree=0.8,\n",
    "        colsample_bylevel=0.7,\n",
    "        objective= 'multi:softprob',\n",
    "        seed=3)\n",
    "\n",
    "modelfit(xgb1, X_train, y_train, cv_folds = kfold)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "随机样本采样的比例选择0.5，虽然只是粗调，但感觉卿老师选的0.3有点小，最后画出的图基本是采样比例越大模型越好\n",
    "因为是多类分问题，且损失函数是log_loss，所以objective使用multi:softprob  \n",
    "可以看出，在最佳的参数n_estimator（其他均为缺省参数）下，训练集上最优的结果logloss为0.4884125621349346（当随机采样比例为0.3时，logloss值为0.5220845701932315）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\ipykernel_launcher.py:1: FutureWarning: from_csv is deprecated. Please use read_csv(...) instead. Note that some of the default arguments are different, so please refer to the documentation for from_csv when changing your function calls\n",
      "  \"\"\"Entry point for launching an IPython kernel.\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEXCAYAAAC3c9OwAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3XmcXGWZ9vHfXdX7ku50FrIvJGEJEAOEXRTZBEZBENEovMLMyOA7DI4zjIOvjjKMKDpuKA6ICowLIKLjgKKyqqBsiawhkISQleykk3TSe9/vH89TSaVT3V2ddHVVp67v53PSdZY6dfepTl31PGczd0dERKS7RL4LEBGRwqSAEBGRjBQQIiKSkQJCREQyUkCIiEhGCggREclIASGSxsz+n5l9P991iBQCBcQQY2Y1ZrbMzD6cNq3WzFaY2YVp0+aY2a/MbLOZNZrZK2Z2vZkNj/MvNbNOM2uKw1Iz+3iOaz/FzFbl8jX6I1M97v5Fd//bHL3eMjM7PRfrzoXBer+G2nYpJgqIIcbdm4DLgRvNbFSc/BVgnrvfC2BmJwK/B/4EHOLu9cBZQAfwtrTVPenuNe5eA1wIfMXMjhyc30T6w8xK8l2DFCF31zAEB+AO4C7gFGATMDZt3hPAt/t4/qXAE92mPQN8OG38XGAB0EgInEPT5h0apzXGZc5Nm3cO8AqwDVgNXA1UA81AF9AUh3E9/F7fAX4dn/80MC2L7XEI8BDwFvAacNHe1ANcC/w4Pm8K4MBlwEpgM3AFcAzwYvzdb0p7nWnAo/H92Aj8BKiP834UX6s5vtanstjGy4B/ja/VCpTE8dXxd3kNOC3DtjgeWAsk06adD7wYHx8LzAO2AuuAr/ewTU8BVvUwrw74IbABWA58FkjEeUnga3EbvAFcGbdjSQ/rWgac3sO8jwFL4vt6X+pvBjDgG8B6YEvcRof39H7n+//rUB3yXoCGvXzjYDiwJv4nvCxtejXQCZzSx/MvJS0g4odeI3BQHD8I2A6cAZQCn4r/Ucvi+BLg/8XxU+N/xoPjc9cAJ6fVeVR83OMHTlodd8QPg2PjB+JPgLv7eE414QP8svico+J2Oay/9ZA5IG4BKoAzgRbgl8BoYHz8gHpnXH563F7lwCjgj8A309a92wdhb9s4bfnngYlAJXBw/D3HpdWXMTyB14Ez0sZ/BlwTHz8JXBIf1wDH97COHt8vQjj8L1Ab61gE/E2cdwXhA3pC3N4PsxcBEf+uNsb3sxz4NvDHOO/dwHygnhAWhxK/JPX0fmvo/6AupiHK3TcTvnlWAb9ImzWc0HW4NjXBzL4S90NsN7PPpi17fJzeRGg9/AhYHOd9EPi1uz/k7u3AVwkfUicSvqHWADe4e5u7Pwr8Cpgbn9sOzDSzYe6+2d3/0s9f7xfu/oy7dxACYnYfy78HWObut7t7R3y9nxO6zQainv9w9xZ3f5DwgX6Xu69399XA48CRAO6+JG6vVnffAHwdeGcv6+1tG6d8y91XunszIfjL4+9S6u7L3P31HtZ9F/H9MLNawrfqu9K2x3QzG+nuTe7+VH82hpklY+2fdvdt7r6M0GK4JC5yEXCju6+Kf6c39Gf9aT4C3Obuf3H3VuDTwAlmNiX+DrWElqO5+0J3X5P2++3L+y2RAmKIMrOLCd/cHga+nDZrM6ErY2xqgrt/ysN+iP8hfMNOecrd6z3sgxgDHAZ8Mc4bR+g6SK2ji/DtdXyctzJOS1ke5wG8n/CBtNzM/mBmJ/Tz11ub9ngHIYx6Mxk4LoZdo5k1Ej5cxgxQPevSHjdnGK8BMLPRZna3ma02s63Aj4GRvay3t22csjJt/hLgHwmtnPXxtcb1sO47gQvMrBy4APiLu6de628IrZdXzexZM3tPLzVmMpLQclyeNi39/R+XXne3x/3Rffs0EbrvxscvJTcRuiPXmdmtZjYsLrqv77dECoghyMxGE/pfPwb8HXCRmb0DwN23E/rtL+jPOt19HeFb93vjpDcJH7yp1zRCV8fqOG+imaX//UyK83D3Z939PEI3zC+Be1Iv05+a+mEl8IcYdqmhxt0/Psj1fCmuc5a7DwMuJnR/pHR/vd62ccbnuPud7v72+Dxn9y8H6cu9QvhwPRv4MCEwUvMWu/tcwvb4MnCvmVVn/2uykfAtfXLatJ3vP6GLZ0LavIn9WHe67tunGhjBrr+zb7n70YQvNgcB/xKn9/R+Sz8pIIamm4BfuvtjsVn9KeB78dsicfyvzeyaGCaY2QRgak8rNLMRhB2ZC+Kke4C/MrPTzKwU+GfCjtI/EwJoO/ApMys1s1MIwXK3mZWZ2UfMrC52m2wldI1A+OY9wszqBmg7pPwKOMjMLon1lJrZMWZ26CDXU0vYAd1oZuOJH1hp1gEHpo33to33YGYHm9mp8X1uIbReOjMtG90JXAW8g7APIrWei81sVGyxNMbJPa7HzCrSB0IL9R7g+niI9WTgnwgtptTv9QkzG29m9YQd630p7fY6JbH+y8xsdvydvwg87e7L4vt7XNxu2+P26Ozj/Zb+yvdOEA39G4D3Eb5Z1Xeb/ghwfdr4ccADhA+ARuBl4HpgRJx/KeE/TuoInvWEPurRaes4n7CzcQvwB+JO3zjvsDhtS1zm/Di9DPgtoatrK/As8Pa0591G6CZopOejmL6QNn4KfezYjssdTDjyaUNc/6OEfRf9qofMO6lL0pZfRdoBAIQPxc+mbZP5cXs+T/jAX5W27HnAivhaV2exjZex+07tWYR9RdsIO/J/lWkbpi0/ifBh/utu038c3+8mwheC9/Xw/FPi7999mE7Y1/XjuL1XAp9j11FMJYQW7ibCUUyfJLQ4rIfXWZbhNb4Q511B2OGe+n0nxOmnEY5camLXEWM1fb3fGvo3WNzYIiI5YWZnA7e4++Q+F5aCoi4mERlQZlZpZueYWUnsavs84QAJGWLUgpAhwcxOBn6TaZ6Ho7CkQJhZFaG77BDCfpJfA59w9615LUz6TQEhIiIZqYtJREQyGnIXABs5cqRPmTIl32WIiAwp8+fP3+juo/pecpchFxBTpkxh3rx5+S5DRGRIMbPlfS+1O3UxiYhIRgoIERHJSAEhIiIZKSBERCQjBYSIiGSkgBARkYwUECIiklHRBMSaLc08+dpqvKur74VFRKR4AuKX3/4XTrhrJi0tzfkuRURkSCiagDh6RrgDYtO2t/JciYjI0FA0AVFSGe5n3rxtS54rEREZGoomIJIxIFqaGvtYUkREoIgCorQqBETbdrUgRESyUTQBUV5dD0DbDt3USkQkG0UTEBU1dQB0NKsFISKSjaIJiMqa0ILobNmW50pERIaGogmIqtoQEK6AEBHJStEERHllLV1u0Kp9ECIi2SiagLBEgu1WgbU15bsUEZEhIWcBYWa3mdl6M3u5h/lmZt8ysyVm9qKZHZWrWlJ2WBUJBYSISFZy2YK4Azirl/lnAzPicDlwcw5rAaDFqkh2bM/1y4iI7BdyFhDu/kegtwsfnQf80IOngHozG5uregBak1WUKiBERLKSz30Q44GVaeOr4rQ9mNnlZjbPzOZt2LBhr1+wLVlNWacCQkQkG/kMCMswzTMt6O63uvscd58zatSovX7BjpIaKrp27PXzRUSKST4DYhUwMW18AvBmLl+wo7SaSgWEiEhW8hkQ9wH/Jx7NdDywxd3X5PIFX9+aoMoVECIi2SjJ1YrN7C7gFGCkma0CPg+UArj7LcADwDnAEmAHcFmuakmZOn4sNcubaWvvoKw0Z7+6iMh+IWefku4+t4/5Dvx9rl4/E6usI2FO09ZGGkaMHMyXFhEZcormTGqAZGW4HlPT1k15rkREpPAVVUCUxntCNG/VfalFRPpSVAFRVj0cgJZtCggRkb4UVUBUDGsAoK1pc54rEREpfEUVEFW1IwDo2NGY50pERApfUQVEdV1oQXQ2KyBERPpSVAFRWRv2QaCAEBHpU1EFhCVL2U6F7ionIpKFogoIgO1WTbJN96UWEelL0QVEc6KG0na1IERE+lJ0AdGSrKGsXS0IEZG+FF1ArG8rJ9mmFoSISF+KLiBq6kdSZ7qrnIhIX4ouIDor6hnmTYSLyYqISE+KLiCssoFhtoNtzS35LkVEpKAVX0BUhbOpt721Ic+ViIgUtqILiNKacD2mpsb1ea5ERKSwFV1AlA8LAbFji24aJCLSm6ILiMphowBo3aouJhGR3hRdQNQMDwHR3qQWhIhIb4ouIGqHjwaga4fuKici0puiC4iSqno6SGAKCBGRXhVdQGBGEzUkWnVPCBGR3hRfQABNyWGUtikgRER6U5QB0VIyjAoFhIhIr3IaEGZ2lpm9ZmZLzOyaDPMnm9kjZvaimf3ezCbksp6UlrIGqjoVECIivclZQJhZEvgOcDYwE5hrZjO7LfZV4IfuPgu4DvhSrupJ11E5kuFdjXR16YJ9IiI9yWUL4lhgibsvdfc24G7gvG7LzAQeiY8fyzA/J7x6NMPZxpbtumCfiEhPchkQ44GVaeOr4rR0LwDvj4/PB2rNbET3FZnZ5WY2z8zmbdiw72dAJ4eNJmnOWxvX7PO6RET2V7kMCMswrXufztXAO83sOeCdwGqgY48nud/q7nPcfc6oUaP2ubDyujEAbNv45j6vS0Rkf1WSw3WvAiamjU8AdvtEdvc3gQsAzKwGeL+7b8lhTQBUN4SA2LFZLQgRkZ7ksgXxLDDDzKaaWRnwIeC+9AXMbKSZpWr4NHBbDuvZadiI0NPV1rh2MF5ORGRIyllAuHsHcCXwO2AhcI+7LzCz68zs3LjYKcBrZrYIOAC4Plf1pKsdORaAribdE0JEpCe57GLC3R8AHug27XNpj+8F7s1lDZlYRT1tlGDbdclvEZGeFOWZ1JixyevYtkk7qUVEelKcAQG0WAXjfV2+yxARKVjFGxC1k6ix1nyXISJSsIo2INorRlLvm3W5DRGRHhRtQHj1KEawlcYdakWIiGRStAGRrD2AEuti00Yd6ioikknRBkR5fepyG6vzXImISGEq2oCoaggnyzXrchsiIhkVbUCkLrfRqsttiIhkVLQBUTMitCBeeHVxnisRESlMRRsQVjmcDpIcVqejmEREMinagCCRoDHZQNkOdTGJiGRSvAEBbC0bQ22rLrchIpJJUQdEW/VYRnRtoL2zK9+liIgUnKIOCB82nrFsYt2WHfkuRUSk4BR1QJQOn0i5dbB+rU6WExHprqgDonr0FAC2rn0jv4WIiBSgog6I+rFTAXjwqfl5rkREpPAUdUBUjpgMwIkjW/JciYhI4SnqgKCqgRbKKdmmfRAiIt0Vd0CY0Vg6isoWnSwnItJdcQcEsL1iLPVt63DXneVERNIVfUB01IzjADaxtbkj36WIiBSUog+IRP0ERtPI6k1b8l2KiEhBKfqAKB8xiYQ5m9Yuz3cpIiIFpc+AMLNpZlYeH59iZleZWX02Kzezs8zsNTNbYmbXZJg/ycweM7PnzOxFMzun/7/CvqkfE86FuOfhJwf7pUVEClo2LYifA51mNh34ATAVuLOvJ5lZEvgOcDYwE5hrZjO7LfZZ4B53PxL4EPBf/ah9QNQeEALi5NHNg/3SIiIFLZuA6HL3DuB84Jvu/klgbBbPOxZY4u5L3b0NuBs4r9syDgyLj+uAN7Mre+DY8Ml0kiCxWZfbEBFJl01AtJvZXOCjwK/itNIsnjceWJk2vipOS3ctcLGZrQIeAP4h04rM7HIzm2dm8zZs2JDFS/dDSTmbS8dQu33ZwK5XRGSIyyYgLgNOAK539zfMbCrw4yyeZxmmdT/ZYC5wh7tPAM4BfmRme9Tk7re6+xx3nzNq1KgsXrp/ttdOYVznara1tA/4ukVEhqo+A8LdX3H3q9z9LjMbDtS6+w1ZrHsVMDFtfAJ7diH9DXBPfJ0ngQpgZFaVD6SGaUy1tSxd3zToLy0iUqiyOYrp92Y2zMwagBeA283s61ms+1lghplNNbMywk7o+7otswI4Lb7OoYSAGOA+pL5VjT2YGmvhCz99bLBfWkSkYGXTxVTn7luBC4Db3f1o4PS+nhR3bF8J/A5YSDhaaYGZXWdm58bF/hn4mJm9ANwFXOp5uOZF/cTDALhwiq7qKiKSUpLNMmY2FrgI+Ex/Vu7uDxB2PqdP+1za41eAk/qzzlwoHT0DgI4Ni/NciYhI4cimBXEdoRXwurs/a2YHAvvXJ+mw8bRZGeVblua7EhGRgtFnC8Ldfwb8LG18KfD+XBY16BIJNldMpGH7Srq6nEQi0wFYIiLFJZud1BPM7H/MbL2ZrTOzn5vZhMEobjC92jaaybzJ6kadUS0iAtl1Md1OOPpoHOFEt/vjtP3KwYcdySRbz8LVb+W7FBGRgpBNQIxy99vdvSMOdwADf7ZanjVMPoxS62TVGwvzXYqISEHIJiA2mtnFZpaMw8XAplwXNtjKxoZDXVtWvpjnSkRECkM2AfHXhENc1wJrgAsJl9/Yv4w6lE6SlG96Jd+ViIgUhGwutbHC3c9191HuPtrd30c4aW7/UlrBChvHpPalrN+mE+ZERPb2jnL/NKBVFIjaKUcyM7GcBW9uzXcpIiJ5t7cBsV+eKFA7eTbjbRNLlq3IdykiInm3twEx6NdLGgzlE2YDsG3583muREQk/3o8k9rMtpE5CAyozFlF+TTmCACSGxbkuRARkfzrMSDcvXYwCykINaPZxHDGtyxh/dYWRg+ryHdFIiJ5s7ddTPut0vGzONSW8/QbOqNaRIqbAqKbmm2vc5CtYv7r3W9+JyJSXBQQ3STO+U9KrZMtS57OdykiInmlgOhu0vEAjN3yPBubWvNcjIhI/mRzue9tZra127AyXgL8wMEoclBVNdBcP4NjE6/xjPZDiEgRy6YF8XXgXwiX+p4AXA18D7gbuC13peVPWUcTRycW8czr6/NdiohI3mQTEGe5+3fdfZu7b3X3W4Fz3P2nwPAc15cXyTP+nVpr5vn5f853KSIieZNNQHSZ2UVmlojDRWnz9sszqpl8AgCzuhaydENTnosREcmPbALiI8AlwPo4XAJcbGaVwJU5rC1/6ibSUTOOYxOv8dAr6/JdjYhIXmRzue+l7v5edx8Zh/e6+xJ3b3b3JwajyEFnRsmBJ3NiYgHfeXRRvqsREcmLbI5imhCPWFpvZuvM7OdmNmEwisurGWfSYNuY1r5Ih7uKSFHKpovpduA+YBzhSKb747Q+mdlZZvaamS0xs2syzP+GmT0fh0Vm1tif4nNq2qm4JTgl8TyPLtTRTCJSfLIJiFHufru7d8ThDmBUX08ysyTwHeBsYCYw18xmpi/j7p9099nuPhv4NvCLfv8GuVLVAKXVnJGYxxd+rduQikjxySYgNprZxWaWjMPFwKYsnncssCTuw2gjnDdxXi/LzwXuymK9g8be/glmJlZQ3rqJVZt35LscEZFBlU1A/DVwEbAWWANcCFyWxfPGAyvTxlfFaXsws8nAVODRLNY7eGacCcA77AU++N2n8lyMiMjgyuYophXufq67j3L30e7+PuCCLNad6bakPZ038SHgXnfvzLgis8vNbJ6ZzduwYUMWLz1AxsyCZBlzy/4IQFfX/nnah4hIJnt7sb5/ymKZVcDEtPEJQE/X0P4QvXQvufut7j7H3eeMGtXn7o+BYwbHfIyjbDFNjRs451uPD95ri4jk2d4GRKbWQXfPAjPMbKqZlRFC4L49VmR2MOGSHU/uZS25NesDJLydi6rmM6q2PN/ViIgMmr0NiD77Wty9g3Cm9e+AhcA97r7AzK4zs3PTFp0L3O3uhdl/M3Y2jDyIS2uf5fHFG3mPWhEiUiR6vCe1mW0jcxAYUJnNyt39AeCBbtM+12382mzWlTdmcMRFjH/sC0ywTbzZWJrvikREBkWPLQh3r3X3YRmGWnfvMVj2S0dcCMC/lf6Yt3a0s2jdtjwXJCKSe7qjXDYapsL00zmjdhkNFfDv9y+gUHvEREQGigIiW8ddQWL7Oj7HLfxpySZ+t0BXeRWR/ZsCIlvTToOGaZw3oZnK0iRX3f0cTa0d+a5KRCRnFBDZSiTAHVv1LOfUL6eto4v/uF/XaBKR/ZcCoj8+/gRUj+ZrI+5n3LByfjpvJb99eU2+qxIRyQkFRH+UVUNZDSx7nD9caFSXJfn7O59j2cbt+a5MRGTAKSD66++fgrpJlD52Hb/9xNsx4OwbH2e79keIyH5GAdFfJeVQUgFrnmfiG/dw+2XH0NzeyUk3PEpbR1e+qxMRGTAKiL1x5TNQUQe/vpqTx3Ry/fmH09jcznFffJiOToWEiOwfFBB7www+9hh4F9x8Ih85dhKf/atD2byjnWOuf5iW9oxXLRcRGVIUEHtrxDSonwQ7NsGL9/C3Jx/I5987c2dIbN7elu8KRUT2iQJiX1z1HJQPg19+HBpXcNlJU5k2qpptLR0c98VHmL98c74rFBHZawqIfZFIwhXx8t83nwgdrTzyz6dw35UnYQYX3vxnbvnD67Rrv4SIDEEKiH01fApc+ANo3QbfnAXArAn1PPOZ06mvKuWG37zKrGsf5InFG/Nbp4hIPykgBsJh58PbPwlNa+HG2QDUVZbyl387g1svOZoudy7+wdMced2DrHxrR56LFRHJjgJioJz6b3DIe2DzG/CtowEwM848bAwvfP5MJgyvZPOOdk7+ymMc98WHebOxOc8Fi4j0TgExUBJJuPA2qBgOby2BF3+2c1ZFaZIn/vVUnvr0aYyqKWPd1lZOvOFRpn/mAZasb8pj0SIiPbOhduObOXPm+Lx58/JdRs/am+GrB0PrFnjfLTB77h6LrNq8g/ff/GfWbW0FCPsqLpjFuw4ZRXlJcrArFpEiYGbz3X1Ov56jgMiB1ib4xkxo2QJnfgFOuDKcXNfNpqZW3nvTE6xpbNl58+9RteXcNPdIjpnSQCKx53NERPaGAqKQdLTC1w8NJ9JVj4JPvBCuBptp0c4u/vT6Jv7pp8+zKZ5gZ8CYugpuv+wYDj6gFssQMCIi2VJAFJquLrjxbbBlBZRWwRVPhDOwe7GjrYOHXlnH5/53AVua24EQFg3VZVz97oM54cARTB5RpcAQkX5RQBSqJQ/DnR8Ed7joh3Doe7J62samVj5w859ZubmZzi7f2Q2VCox/PesQjj9wBBMbKhUYItIrBUQha1wB/3UCtDVB7Vi48lkor8366e7O6xu28+TSTXzjoUVs3t5G+jtXV1nKJcdP5vDxdRwyppZJDVXahyEiOykgCl1HKzz0eXj6ZsBg9Ez4v3/eq1W5O0vWN/GxH85jdWNoYXR1eysTBsmEMbauki9dcASTGqoYW1dBSVJHN4sUm4ILCDM7C7gRSALfd/cbMixzEXAt4MAL7v7h3tY5pAMiZcVTcPvZ4XLhb/swvPt6qGrY59W2tHfy2tptvLZ2G19/6DU2NLXRldY1lWIGCTMaqsu49MQpTGqoYmJDFROHV9JQXabuKpH9UEEFhJklgUXAGcAq4Flgrru/krbMDOAe4FR332xmo919fW/r3S8CAqC9Bb59NGxdBYnScDjsnMvCHesG2IZtrSxet43P/PIlVm9upsuhyx139ggPCC0PMyNhcEBtBVedPoOxdRWMratgTF0l1WVJhYjIEFNoAXECcK27vzuOfxrA3b+UtsxXgEXu/v1s17vfBETKmhfhjveEE+uS5fDeb8KsD4YzswfB9tYOVm1uZsVbO/jCr17hzS3NeB8BAmFHucUgMQun5I+pq+SfzzyIkTXl1FWWUldZyrDKUmrLS7Q/RCTPCi0gLgTOcve/jeOXAMe5+5Vpy/yS0Mo4idANda27/7a39e53AQHh6Kalj8FPLwk7sUur4ILvwSF/lfEEu8HU2tHJ+q2trNnSwpotzXzq3hdp6+gimTCcsC+ktyBJZ5YKFsOAzi6nJGlcdeoMhu0MlJLws6J057TykoRaLCL7qNAC4gPAu7sFxLHu/g9py/wKaAcuAiYAjwOHu3tjt3VdDlwOMGnSpKOXL1+ek5rzzh0W3ge/+DvoaIayWph7J0w5Oe9B0Rd3Z0tzOxubWtnY1MaW5na+/JtXWd3YHEIEdgVJHO++U703qRYL7AqYVOBgxri6Cv7h1BlUlyepKiuhpqKE2vLws6a8hOoytWKkuBVaQGTTxXQL8JS73xHHHwGucfdne1rvftmC6K6zA164E+67CvBwBvZZN8DhF0JZVb6rG3BtHV1sbWlna3M7W5rb2drSwX/cv4CVm5tp6wg3WypJ2s6ASQUOWbZcukvlRCpcUrFhBh2dTmnS+IdTZ1BZlqS8NEllHCpKE1SWJiktSVCaTFCSMMpKws/SZJhWVpKgNBmmlyYSCiUpGIUWECWE7qPTgNWEndQfdvcFacucRdhx/VEzGwk8B8x29009rbcoAiKlvSXcqW7zG+GIp0QJHHcFzPnrPs/ILhbuzva2Tna0ddDc1smOtk62t3bQFIevPfgabza24O60d4a/9VT3WKolA+xT4GTL0h6kHqdaUcn0JlL35QndcRCCsqf1tnc6pSUJPnHaDJIJI2lGMmGUJG2P8YQZJYlEGE/E+d0eh/FEt/Hslk0d6CCFo6ACAsDMzgG+Sdi/cJu7X29m1wHz3P0+C39BXwPOAjqB69397t7WWVQBkeIOy/8E93wUdsQ701UMhwu+C9NPH7Qd2sWkvbOLlvZOmts7aWnroqWjk+a2MN7e2RUHp6PTdxtv7+yiraOLts4ubnx4Ee2dvtuHevp/N2fXB38yYbsFVjpPe551mz6U7BEXaS25VFD21uDKtMyu0A2PdtuePdTQkQrbPlp36euqqyzlxOkjd+/aZFd3Z2rlf1y0AYBTDh69e7coFg/q2P15YV27B2rqMPTuB4IYxrsPO4AjJw3vte6eFFxA5EJRBkS6rWtg/h3wxNegsz0cFvuuz4Qjn2rH5Ls6ybOuLqejy+nscjq6uujqgo6urjgepnd/nFq2s/t8dzo705Z1p7Ori47O9HHnB4+/gRMOVli3LVzCPpVoqVZb95aP7/wn6Ojjgx36+PCPr9e5l0HT23K9NYQ8fZkePkrTL5HT43p6L2WnqSOqeOxf3pXl0rtTQBSTjjZ49X74+cfAO8O0qe+EIz4Ah74XKuvzW5+IFBRpxKPtAAAQEElEQVQFRLHasAh+fAFsWUX4LmLhgoBHfABmvBtKK/JdoYjk2d4EREmuipFBNOog+OTLob27+i/w0s/g2e/DwvvD/OrR8J6vw9R3QEVdfmsVkSFDLYj9VVcnvPFHeOleeOGuXd1Q5cPgpKvCzu0xb4OELtwnUgzUxSSZdbbDqmfDfSmevgXatofpiVI47HyYfhpMOxVqRue3ThHJGQWEZKdpA7z+KDz4Gdi+kZ3HUJRVw7F/F1oXE4+FZGleyxSRgaOAkP7r6oK1L4bWxWNf3NUVZUk46KzQuph+GgyfktcyRWTfKCBk37VsDfsuljwMz98JnfG4dix0QZ35BZjydhg2Lq9likj/KCBkYLnDpiWw5JEQGot/G3Z+A5RUwKyLYPLbYdLxUD+p4C8oKFLMFBCSW12dsPYlWPYEPP5VaN68a16yDA55TwiLScfDAYfrEiAiBUQBIYOrqxPWvxJuofr7L8frRKX9PVkS3nF12OE94RidgyGSRwoIyb/GlSEwHvwsbF8frkKbUlYNR18WwmLcbKifrG4pkUGiM6kl/+onhmHWB8J46zZYPT+ExpP/BU/elLawhVbFcX8H4+fA+KOhekReyhaRPakFIYOroxXWLYA3nwvDyz+H9h275pdUhIsNjj86hMbYWeGKtSKyT9TFJENTa1MIi9XzYNU8WPwgdLbtmm8JOPpSGDsbxr4NRs+EkrK8lSsyFKmLSYam8hqYenIYUrasDoGxej7MuwPm3bb7c8qqwy1Yx74NxsyCA2aGaSIyYNSCkKGhqyvcenXNC7DmeZj/39DSuPsyloBDz4UxR+waasdqR7gI6mKSYuMOjStg3cvh/Iynb4Xmt9jtUNtECUw+EQ5IhcbhMPJgdVFJ0VFAiEC4XMi6BSE01r0EL/8C2prSFrBwIt+Yw0NoHBB/VjXkrWSRXFNAiPSkswPeej2ExtqX0rqo0v/+LdyqdfZHYOQMGDEDRkwP16BSN5UMcQoIkf5q2hBaGWtfgrUvw8v37n5yX0pZDRx8Trh736hDYfSh4Qq3upyIDBEKCJGB0NUZ7u+9aTFsXBIuWLhpMSx/Mu3qtlFpVbh/xojpu1odI6dD5fD81C7SAx3mKjIQEkkYPjkM00/ffV7rNtiwCDYshA2vwnM/gVd/laHVYVBeCzPPjaERw2P4FO0glyFDLQiRgdDZDpuXx1bH4vDzuZ/sugFTupIKmPrOGBrTd7U+ag7Qvg7JGXUxiRSi5kbY9HoIjU1LYN7t4XDc7q0OS4ZLi6R2jo+cHh9P00mAss8KrovJzM4CbgSSwPfd/YZu8y8F/hNYHSfd5O7fz2VNIoOush4mHB0GgFM/G352dcHWVSE0Nsb9HC/eA28+z+5HVxHutzHphN33c4yYDnUTtaNcciZnLQgzSwKLgDOAVcCzwFx3fyVtmUuBOe5+ZbbrVQtCikJ7865WRyo8Xv11t/M5otSO8pEzoOFAaJgWfurwXElTaC2IY4El7r4UwMzuBs4DXun1WSICpZXxRL7Dd5/uDts3xP0cS3YFyPpXYOF9uy9ryXCNqlRojJwBIw8KP3XzJslCLgNiPLAybXwVcFyG5d5vZu8gtDY+6e4ruy9gZpcDlwNMmjQpB6WKDBFmoWVQMxqmnLT7vM72cOmRt94IJwW+tTQMi34HHS27L5sohfFHhaOqUkP95PCzdiwkEoPz+0hBy2VAZGrbdu/Puh+4y91bzewK4L+BU/d4kvutwK0QupgGulCR/UKyNOzQHjEN6HZ4bmcHbF4GGxeFYdNieOV+WPkMe/63JFz4sKIOjrgIGqbC8KmhJTJ8su7PUURyGRCrgIlp4xOAN9MXcPdNaaPfA76cw3pEileyJOzYHjkdOCdMO+874WdHG2xZGQIkNTz/E2jeDM98N8O6ysN9xtODI/W4Ytjg/D4yKHIZEM8CM8xsKuEopQ8BH05fwMzGuvuaOHousDCH9YhIJiVlaS2P6Mz/CD/dYcem0G21+Y3YbRV//uVH7NH6SJSG+403HBjDY2rsupoMNWPUdTXE5Cwg3L3DzK4Efkc4zPU2d19gZtcB89z9PuAqMzsX6ADeAi7NVT0ishfMoHpkGCYes+f81m2Zw2PBL/e8LAmEkwQnnQD1k0Jo1KeGSTrqqgDpRDkRyY2O1nB2eeMKaFwWf66AxQ9B23Yy7vsYeVAIi1RopAdJ1QgFyD4otMNcRaSYlZTHq98elHl+a1PY99G4IgZJDJPXH4Elj9DjzvPpZ8TQ6BYklcMVIANMASEi+VFeEy6bPvrQzPNbtu5qdaTCY/NyWPp7WPy7zM8prYID39Wt9RGDRDvQ+00BISKFqWJY5pMFU5o3p7U+0oJk6WPQviPzc8qqYdqpu+/7SA3lNbn7XYYoBYSIDE2Vw8Mw9m17znOHHW/Flsfy3YPk1V9nvikUFgJk+ukZdqJPDGe3FxkFhIjsf8ygekQYxh+15/zUJUvS932kfi76HXQ07/mc1CG8dRNjq2Mi1E3a9Xg/vOKuAkJEik/6JUsyHb7b1QVN63YPj1SYvPbAnpcuCSuNXVjvCsFRNwHqxsOw+LN69JA7D0QBISLSXSIBw8aGYdLxe87v6oKmtdAYj8LasmLX48UPZ26BQDgLffzRMTjiUDc+hMmwCVDVUFBHYikgRET6K5GAYePCMCnDNUhT+0C2roItq2FrHFKPF94XzhPJpKQyrHPYhF2tkFSADPK+EAWEiMhAS98HkmknOoRWyPYNu0Jky6oYIqvCeSBtf2CPc0GGHwifeC7n5acoIERE8iGRgNoDwjD+6MzLdLTBtjW7gmNchh3uOaSAEBEpVCVl4XDb4ZPz8vJDa5e6iIgMGgWEiIhkpIAQEZGMFBAiIpKRAkJERDJSQIiISEYKCBERyUgBISIiGQ25e1Kb2QZg+V4+fSSwcQDLGQxDsWYYmnWr5sGhmgdH95onu/uo/qxgyAXEvjCzef29aXe+DcWaYWjWrZoHh2oeHANRs7qYREQkIwWEiIhkVGwBcWu+C9gLQ7FmGJp1q+bBoZoHxz7XXFT7IEREJHvF1oIQEZEsKSBERCSjogkIMzvLzF4zsyVmdk2+6+mJmS0zs5fM7HkzmxenNZjZQ2a2OP4cnucabzOz9Wb2ctq0jDVa8K243V80s8G9JVbvNV9rZqvjtn7ezM5Jm/fpWPNrZvbuPNU80cweM7OFZrbAzD4Rpxfstu6l5kLf1hVm9oyZvRDr/vc4faqZPR239U/NrCxOL4/jS+L8KQVU8x1m9kbatp4dp/f/78Pd9/sBSAKvAwcCZcALwMx819VDrcuAkd2mfQW4Jj6+Bvhynmt8B3AU8HJfNQLnAL8BDDgeeLqAar4WuDrDsjPj30g5MDX+7STzUPNY4Kj4uBZYFGsr2G3dS82Fvq0NqImPS4Gn4za8B/hQnH4L8PH4+P8Ct8THHwJ+WkA13wFcmGH5fv99FEsL4lhgibsvdfc24G7gvDzX1B/nAf8dH/838L481oK7/xF4q9vknmo8D/ihB08B9WY2dnAq3aWHmntyHnC3u7e6+xvAEsLf0KBy9zXu/pf4eBuwEBhPAW/rXmruSaFsa3f3pjhaGgcHTgXujdO7b+vUe3AvcJqZ2SCVC/Rac0/6/fdRLAExHliZNr6K3v9o88mBB81svpldHqcd4O5rIPwHBEbnrbqe9VRjoW/7K2Nz+7a0rruCqzl2YRxJ+JY4JLZ1t5qhwLe1mSXN7HlgPfAQoTXT6O4dGWrbWXecvwUYMbgV71mzu6e29fVxW3/DzMq71xz1ua2LJSAyJXuhHt97krsfBZwN/L2ZvSPfBe2jQt72NwPTgNnAGuBrcXpB1WxmNcDPgX909629LZphWl7qzlBzwW9rd+9099nABEIr5tBMi8WfBVF395rN7HDg08AhwDFAA/CvcfF+11wsAbEKmJg2PgF4M0+19Mrd34w/1wP/Q/hDXZdqCsaf6/NXYY96qrFgt727r4v/wbqA77Gra6NgajazUsIH7U/c/RdxckFv60w1D4VtneLujcDvCf309WZWEmel17az7ji/juy7MAdcWs1nxW4+d/dW4Hb2YVsXS0A8C8yIRySUEXYq3ZfnmvZgZtVmVpt6DJwJvEyo9aNxsY8C/5ufCnvVU433Af8nHkFxPLAl1T2Sb936X88nbGsINX8oHqkyFZgBPJOH+gz4AbDQ3b+eNqtgt3VPNQ+BbT3KzOrj40rgdML+k8eAC+Ni3bd16j24EHjU457gwdJDza+mfXkwwj6T9G3dv7+Pwd7znq+BsAd/EaFf8TP5rqeHGg8kHNHxArAgVSehb/MRYHH82ZDnOu8idBO0E76V/E1PNRKatd+J2/0lYE4B1fyjWNOL8T/P2LTlPxNrfg04O081v53QBfAi8Hwczinkbd1LzYW+rWcBz8X6XgY+F6cfSAisJcDPgPI4vSKOL4nzDyygmh+N2/pl4MfsOtKp338futSGiIhkVCxdTCIi0k8KCBERyUgBISIiGSkgREQkIwWEiIhkpIAQEZGMFBAiWTCz2d0uUX2uDdBl483sH82saiDWJTKQdB6ESBbM7FLCiUVX5mDdy+K6N/bjOUl37xzoWkTSqQUh+xUzm2LhZjXfizdReTBehiDTstPM7LfxyrmPm9khcfoHzOzleCOWP8bLs1wHfDDegOWDZnapmd0Ul7/DzG62cKOcpWb2znjF0oVmdkfa691sZvNs95u7XAWMAx4zs8fitLkWbhr1spl9Oe35TWZ2nZk9DZxgZjeY2Svxqp1fzc0WlaKWj9PaNWjI1QBMATqA2XH8HuDiHpZ9BJgRHx9HuJ4OhMsQjI+P6+PPS4Gb0p67c5xwg5a7CZcyOA/YChxB+AI2P62W1CUxkoQLq82K48uIN4kihMUKYBRQQrhswvviPAcuSq2LcGkKS69Tg4aBHNSCkP3RG+7+fHw8nxAau4mXoz4R+Fm8nv53CXdDA/gTcIeZfYzwYZ6N+93dCeGyzt1f8nDl0gVpr3+Rmf2FcP2cwwh3U+vuGOD37r7Bw30GfkK4Gx5AJ+EqqRBCqAX4vpldAOzIsk6RrJX0vYjIkNOa9rgTyNTFlCDcDGZ29xnufoWZHQf8FbDznr5ZvmZXt9fvAkrilUqvBo5x982x66kiw3p6uytZi8f9Du7eYWbHAqcRrk58JeHuZyIDRi0IKUoebmLzhpl9AHbe0P1t8fE0d3/a3T8HbCRcQ38b4R7Le2sYsB3YYmYHEG4IlZK+7qeBd5rZSDNLAnOBP3RfWWwB1bn7A8A/Em7EIzKg1IKQYvYR4GYz+yzhfr53Ey61/p9mNoPwbf6ROG0FcE3sjvpSf1/I3V8ws+cIXU5LCd1YKbcCvzGzNe7+LjP7NOE+BAY84O6Z7v9RC/yvmVXE5T7Z35pE+qLDXEVEJCN1MYmISEbqYpL9npl9Bzip2+Qb3f32fNQjMlSoi0lERDJSF5OIiGSkgBARkYwUECIikpECQkREMvr/AUgucUrym+EAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "cvresult = pd.DataFrame.from_csv('1_nestimators.csv')\n",
    "        \n",
    "# plot\n",
    "test_means = cvresult['test-mlogloss-mean']\n",
    "test_stds = cvresult['test-mlogloss-std'] \n",
    "        \n",
    "train_means = cvresult['train-mlogloss-mean']\n",
    "train_stds = cvresult['train-mlogloss-std'] \n",
    "\n",
    "x_axis = range(0, cvresult.shape[0])\n",
    "        \n",
    "pyplot.errorbar(x_axis, test_means, yerr=test_stds ,label='Test')\n",
    "pyplot.errorbar(x_axis, train_means, yerr=train_stds ,label='Train')\n",
    "pyplot.title(\"XGBoost n_estimators vs Log Loss\")\n",
    "pyplot.xlabel( 'n_estimators' )\n",
    "pyplot.ylabel( 'Log Loss' )\n",
    "pyplot.savefig( 'n_estimators4_1.png' )\n",
    "\n",
    "pyplot.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "由上图可知训练集和测试集上，随着弱学习器数量的增加，log_loss逐渐减下降，测试集上的logloss在220左右达到收敛  \n",
    "一开始弱学习器数量很少，此时训练集和测试集上的logloss基本相等，且都很大，只是因为模型太差，属于欠拟合状态  \n",
    "随着弱学习器数量增加，训练集和训练集上的logloss均下降，测试集logloss逐渐稳定，而训练集logloss则继续下降，两个差距逐渐拉大，此时渐渐变成过拟合  \n",
    "理论上随着弱学习器继续增加，训练集上的logloss可以达到0，正确率达到100%，但我们更关注模型在测试集上的性能"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\ipykernel_launcher.py:1: FutureWarning: from_csv is deprecated. Please use read_csv(...) instead. Note that some of the default arguments are different, so please refer to the documentation for from_csv when changing your function calls\n",
      "  \"\"\"Entry point for launching an IPython kernel.\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA2cAAANGCAYAAAB5u/b2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAPYQAAD2EBqD+naQAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzs3XmcXFWZ+P/Pk4SwdyMwEMIWEEQQBQQUEAVBxAFxGZ1R0RlR0RFHB9TRL6A/DCigjsOioIyCoo6K+wKIgCwKAUFANsMuIBAIhKUDCUlnOb8/7q10pVJbV1f3reXzfr3qlep7zz33qeol/fQ55zmRUkKSJEmSVKxJRQcgSZIkSTI5kyRJkqSOYHImSZIkSR3A5EySJEmSOoDJmSRJkiR1AJMzSZIkSeoAJmeSJEmS1AFMziRJkiSpA5icSZIkSVIHMDmTpDIR8e2IWBwRL61y7uiISBFxSMXxgfzcdRHxTEQsiYi5EfG7iDg0IlYvazsj76P8MT8ibomIoyJi8kS8znoi4iMRcVjRcYxFRKwVETMjYt8q5w7L3/cZBcQ1PY9r54m+d5Ei4tyIeK7oOADy9z9FxIZFxyJJlUzOJGllRwGPAd+NiNVKB/Nk7Xjg3JTS+WXHtwX+AnwGuBr4N2A/4GPAI8C3gc9Wuc/XgD3zx78As4BTgS+3/yWN2keAw4oOYozWAj4H7Fvl3IVk7/ujExlQbjpZXH2VnEmSmjOl6AAkqZOklOZHxAeAS8iSqs/lSdr3gblkyRsAETEF+BWwPvCKlNIdFd39JCJOAHapcqu/p5T+VPbx7yJiR+BdwCfb9oK0ipTSE8ATRcfRThGxVkppYdFxSJLGxpEzSaqQUvo9cBZwbETsCswEdgI+kFIaKmv6VmAH4MQqiVmprwdTSr9q8tZDwJLyAxExKSI+HRF35tMtH4+I70XEZpUXR8T78+mRiyLiqYj4ZURsX9Fm64g4LyLm5P3NjYjLStPsIuIB4CXAPmXTLh+oF3Te5oyI+NeIuCMiFuZxvLHJ113e19SI+GzZ630iIr4TEf9Q0W6/iLgyIp6MiOcj4u8R8fN8OuMMRpKvz5W9jnPza1eZ1pj3dXtE7BkR1+R9PhAR78vPHxwRN+Wv7baIeENFPNvkcd6Tt3kkIs4vnx6bT7H8c/7hd8rimlnW5k0RcW3ex7MRcWlE7Flxr9K0vJdHxM8i4mngvvxc3c9vjff8qLy/baqc+1JEDJemAEbELhFxQf51uDi/z4XVvh5bERF75/E+m78H10TEwTXaXZt/rT8SEZ+PiMMrP69jjGXHiPh1RDyd3+fmiHhvRZtJ+dfrXfnXzDMRcWtEHFnW5h8i4psR8VDZ1/SsiHhdO+KU1FscOZOk6j4FHAj8DNgcOCuldGlFmwPyf3/TQv+TIht5AxgE3gy8AfhSRbtvAB8CzgAuAGYAnwf2jYiXp5TmAUTEMcBJwI+AY4ANyJLKayNi95TSPXl/vwUmA58G/g5sCOwFrJeff2v+mofIpjcCLG7i9RwM7A4cBzyX9//LiNgupfS3Jq4nIiYBvwZeTTa98xpgS7LppFdGxG4ppefzX74vBK4C3g88A2xK9v5NJZuu+Abgd8A5wNn5LRqNlk0DvpPf+2GyqanfjojNgbeTvb9D+Wv8VURsnVKak187HXgSODq/z/rAe4HrImKXlNJdwE3A+/J7fCF/DeT3IiIOBX5ANmr7LmD1/H28MiL2TyldXRHvL4DzyP6QsHZ+rNHnt5r/I/u6O4yyKbiRrX98D3B+SmleRKwNXArcD/wH2UjyNOC1wLp1+m9KROyT938r8AGyr7uPAOdHxLtSSj/O270sb3c32Xu8EPhwHmtbRMR2ZF9/jwP/Sfa5fQ9wbkRsnFIqTT/+NNn32ReAPwKrAS9m5ff7+8DLyaY+352feznZ96gkrSyl5MOHDx8+qjzIfkFOZL/sr1Pl/EX5+dUrjgfZH79Kj8ll52bk11R7fKei7Yvz42dW9P+K/PiJ+cfrkf2CemFFu82BRcAP8o83yK87ssHrvh24chTvUyJbp7du2bGNgWXA0aPo5515X/9UcXy3/PgR+cdvyz/eqU5fG+ZtZlY5d1h+bkbZsSvzY7uWHVsfWJq/t9PLju+Ut/1YnftPJvtF/W7glCqv5bCK9pPI1ijeCkwqO74OWRI0q+zYzLyP4yv6aOrzWyPenwMPVdz7H/P+3ph/vGv+8Ztb6P9c4LkGba7NX+s6ZccmA7flsUV+7CdkfwDYsOL9+2vl57XGfUrv34Z12vwo/97ZvOL4b4EFwGD+8fnAXxrc71ng1NG+Zz58+OjPh9MaJamKfBTnY8ByYCOyX8ibdSTZ9MTS45YqbU4nG2nanWzk4ViywiA/Kmvz2vzfc8svTCldD9wB7J8f2hNYs0q7h4DLy9o9RTb97VMR8Yl8ilq7/h+4IqX0bNm955KNOmw5ij7eSDYKdn5ETCk9gJvJkr9983Y3A8PANyPivRGxdTteAPBoSunG0gcppafIXsPNaWSEDLL3HspeWx7rsRExOyKGyZK6YWBbYKWppTVsRzb69v2U0vKyGJ4jS5z2iIi1Kq75ecXHY/n8fgfYDCifavc+svf9ovzje4GngS9FxIcjYocm+24oH5V7JfCz/DUDkFJaRjbytBnZewSwD3B5ykeN83bLyZK2dtkPuCz/Hip3LlmxmdJU0+uBnSLi6xFxYEQMVOnreuCwfPrjHlFWaEiSKpmcSVJ1/0X2C9ihwD1k09vWrGjz9/zfygTkh4wkXjfV6P/hlNIN+ePKlNLJZNMV/zkiDszblKY9VasqOKfsfFPtUkqJLFG7mGw61k3AExHx1YgY67S0J6scW0yWNDZrY7JRwGFWTm6XkE2f2xAgpXQfWRLxOHAmcF9E3Fe+zqdFT1U5Nlx5PKU0nD9do+zwKWSfv18Bh5AlGruTJebNvAeNPoeTgBdUHF+p7Rg/vxfl/ZXW2L0AeBPwvTxBImXrLfchS45PAv6arzk7vg0JxwvIRpxrvX5Y+et9bpV21Y61aoMmYzmZ7GfFHmTv4ZP5mrndyq55B/Bd4HCy0cGnIls3Oq2N8UrqESZnklQhHxE4gewX0x+TTYPbBjixomlpDdqbyg+mlB4vJV5kU5qadWv+b2mUrpTwbFKl7XRg3ijbkbICJR9IKU0jG4k4lWxdz3+PIs7xMo/stexe41FaA0dK6aqU0iFk6/X2IPul97SIeOdEB517D9nXy7EppYtTStfnn/9m99Jq9DlcTjZqVS5VNmz181s2QvWWiFiP7I8Sq5ONqJW3uy2l9E6y5GRn4Mdka/DGWmH0abLXWOv1w8pf7xtXadfOZOfJZmJJKS1NKZ2SUno52TTYd5FNJ764NNKZUpqXUjoqpTSD7A85xwD/RMVItySByZkkrSSfRvddsl++jgRIWcn7U4AjI+JVZc1/Ccwmq+r44jbcvlRR7/H838vzf1cqdBARu5NNlbssP3Qt8HyVdpuRT8+qdrOU0t0ppS+Qrel5edmp0Y54tcsFZL/0Ty4bVSx/3FV5QUppWUrpOrICFTDyOkpFTCbqdSQqCqfkVQY3rWhXK667yNacHRoRUdbH2mRr7K5NoyyVX+fzW8t3yEYD30X2B4lrU0p31ug7pZRuSSl9nGwqajP914t1AXAd8E/lI9T5tMz3kBVNuTs//AdgvyjbRDpv989jiaHCZfk9plcc/zeyNYh/qrwgpfRMSulnZKO565OtL61s8/eU0hlkf9gZ03smqTdZrVGSVnYMWdGGf0wpPVN2/P8jm6727YjYOaX0fEppWUS8hWwa2fUR8S2ywhJPk03PeyXZKFi1MvtbRMQe+fO1yaZQHgM8SFaFj5TSXRHxTeBjEbGcbNrUDLLpcw+RjYqQUnomIj4PnBQR3yNbt7YB2WbHi8iqHZaq3J0B/JRsquYwWfL2MuCLZbHdBrwzIt4B/A1YlFK6bTRvYovOA94N/DYiTidbq7OEbL3Ra4Ffp5R+GREfzuO+kGxq6RpkVRsBfg+QUno2Ih4E3hwRl5FNTZyXUnpgnGK/gGxd0Z1kI6C7klX8fLii3X1kifS7I+IOssIWc1JKcyLi02TVGi+IiP8lG7n6FNnX0tGNAhjF57eqlNKdEXEt2dfh5mRVQsv7fyPZKNyvyL4ugmwEaD1GRpHrmRwRb69yfEFK6aL8vpcCV0TEV/L4PwLsCLwrn7YJ2Qj2IcBlEXEi2fv5YUYqVi6nOYdExCoj23mCdTzZGsgrItur8Cmyr82DgU/nUzyJiPPJCujcQFalc0uyvRAfBO6JiEHgCrKpzneSjaTvTlZN9BdNximpnxRdkcSHDx8+OuVBlkgNA9+scX4PsgqEp1QcHyD7xfJ6RvYqm0tWEv0jwFplbWewapXG58lGTk4FplX0PYls/dBdeWxPkBdIqBLfB8jWOC0mG834FbBD2fmNyEZHSknBs3n7o1i5SuSWZAnn/Dy+Bxq8bwk4o8rxB4BzR/k5mEI2Re7m/H15No/3LGCbss/DL/L+F5GNcl4JHFLR1/5k664W5TGemx8/jOrVGm+v8RouaPSayRKUs/PP+wKyMv975/1eWXHtO/PXNExFRUmyLRX+lL/258iSzb0qrp9JlWqDzX5+G7z/H8z7XggMVJzbjizJuDc//wzZaNd7m+j3XGpXKX2grN3eZKNWz+X3uJa8WmRFf3vn79MisrVhXyb7PknklRTrxDKzTiyprN2OZNtkPEP2PXUzq1bZ/AQwi+z7cjFZUnY2sGV+fnWy7TBuIfvZsJAsSZtJ2c8FHz58+Cg9SmVpJUmSulZEXEKWcL+o6FgkqVVOa5QkSV0lIk4B/kI2vXd9simHB5CNHktS1zI5kySNu7zQSj3LU9n+XlIDk8kqqk4jm444G/jXlNL/FRqVJI2R0xolSeMqImYA9zdodnxKaea4ByNJUgdz5EySNN7mkFWoa9RGkqS+5siZJEmSJHUAN6GWJEmSpA7gtMYWRUQA08n2kZEkSZLU39YF5qQxTE00OWvddODhooOQJEmS1DE2Ax5p9WKTs9Y9C/DQQw8xMDBQdCySJEmSCjJ//nw233xzGOOsOpOzMRoYGDA5kyRJkjRmFgSRJEmSpA5gciZJkiRJHcDkTJIkSZI6gMmZJEmSJHUAkzNJkiRJ6gAmZ5IkSZLUAUzOJEmSJKkDmJxJkiRJUgcwOZMkSZKkDmByJkmSJEkdwORMkiRJkjqAyZkkSZIkdQCTM0mSJEnqACZnkiRJktQBTM4kSZIkqQOYnEmSJElSBzA5kyRJkqQOYHImSZIkSR3A5EySJEmSOoDJmSRJkiR1AJMzSZIkSeoAJmeSJEmS1AFMziRJkiSpA5icSZIkSVIHMDmTJEmSpA5gciZJkiRJHcDkTJIkSZI6gMmZJEmSJHUAkzNJkiRJ6gAmZ5IkSZLUAUzOJEmSJKkDmJx1uYXDS5lx9IXMOPpCFg4vLTocSZIkSS0yOZMkSZKkDmByJkmSJEkdwORMkiRJkjqAyVmXW7Rk2Yrn855bXGAkkiRJksbC5KzLrbHa5BXPb3zw6QIjkSRJkjQWJmc9xORMkiRJ6l4mZz3khgdMziRJkqRuZXLWQ+6a+yxDzy8pOgxJkiRJLTA56yEpwY0PPlV0GJIkSZJaYHLW5daaOoUHvngwb3v5pgC8/9wbWDi8tOCoJEmSJI2WyVmP2G3GC4oOQZIkSdIYmJz1iF23HEnOnh9eVqelJEmSpE5kctYjNl1vzRXPb3n4mQIjkSRJktQKk7MeERErnltSX5IkSeo+Jmc9yM2oJUmSpO5jctaDbn7oGYaXLi86DEmSJEmjYHLWgxYvXc5tjwwVHYYkSZKkUTA561HX3+9m1JIkSVI3iZRS0TF0pYgYAIaGhoYYGBgoOpwVzrn6fj5/wewVH88+4UDWmjqlwIgkSZKk3jZ//nwGBwcBBlNK81vtx5GzHvOKGesXHYIkSZKkFpic9ZjtN1mXtVefXHQYkiRJkkbJ5KzHTJk8iV02f0HRYUiSJEkaJZOzHrTbDJMzSZIkqduYnPWgV241su5s6TL3O5MkSZK6gclZD9px08EVz93vTJIkSeoOJmc9aPKkWPH86nufLDASSZIkSc0yOetx37jyPmYcfSELh5cWHYokSZKkOkzOetBaU6dwzdH7FR2GJEmSpFEwOetR09dbk63/Ye2iw5AkSZLUpMKTs4j4SETcHxGLIuLGiHh1g/brRcSZEfFofs0dEXFQ2fmZEZEqHo9V9BF5uzkR8XxEXBkRLxmv11iUvbfZsOgQJEmSJDWp0OQsIt4BnAacCOwCXAVcFBFb1Gg/FbgUmAG8HdgO+CDwSEXTvwKblD1eWnH+08AngI8CuwOPAZdGxLpjflEdZK8XbrDieUqpwEgkSZIkNVL0yNkngHNSSmenlO5IKR0FPAQcUaP9+4H1gbeklGallB5MKV2dUrqlot3SlNJjZY8nSiciIoCjgBNTSr9IKd0OvBdYCzi03S+wSLvPGNnv7P55CwqMRJIkSVIjhSVn+SjYrsAlFacuAfaqcdmbgGuBMyNibkTcHhHHRsTkinbb5lMW74+I8yJi67JzWwHTyu+bUloM/KHOfYmI1SNioPQAOn6Ubc2pI2/LLEvqS5IkSR2tyJGzDYHJwNyK43PJkqdqtiabzjgZOAj4AvBJ4DNlba4D/g04kGzK4zTgmogozfEr9T2a+wIcAwyVPR6u07bjzLp3XtEhSJIkSaqj6GmNAJWLoaLKsZJJwOPAh1JKN6aUziNbr7ZiGmRK6aKU0s9TSrellH4PHJyfeu8Y7gtwMjBY9tisTtuO88d75rnfmSRJktTBphR473nAMlYdrdqIVUe1Sh4FlqSUlpUduwOYFhFTU0rDlReklBZExG3AtvmhUuXGaXl/zdy3NPVxcenjbOmaJEmSJLVHYSNneSJ1I3BAxakDgGtqXDYL2CYiyuN+EfBotcQMsrViwPaMJGL3kyVoB5S1mQrsU+e+XWmtqVN44IsH8+adpxcdiiRJkqQGip7WeApweES8PyK2j4hTgS2AswAi4nsRcXJZ+28AGwCnR8SLIuJg4FjgzFKDiPhKROwTEVtFxCuBnwEDwHcBUlZT/jTg2Ih4a0TsCJwLLAR+OM6vtxCv2maDxo0kSZIkFarIaY2klH6cF+o4jmw/stuBg1JKD+ZNtgCWl7V/KCJeD5wK3Eq2v9npwJfKut0M+BFZwZEngD8Be5T1CfBlYE3g68ALyIqIvD6l9GzbX2QH2HPrkeTsiWcXs+UGhX7aJUmSJFURbk7cmryc/tDQ0BADAwNFh1PXwuGl7HDcxQAc/6YdeO9eWxUckSRJktQ75s+fz+DgIMBgSml+q/0UPa1RE+zS2Y8XHYIkSZKkKkzO+szV91pSX5IkSepEJmeSJEmS1AFMziRJkiSpA5ic9YHSfmc/P2LPFcec1ihJkiR1FpOzPvLiaeuueD7r3icLjESSJElSJZOzPhIRK55fMntugZFIkiRJqmRy1u2GF8DMwewxvKDpy/5w1xMsXrpsHAOTJEmSNBomZ33qucVL2e6zv7OsviRJktQhTM4kSZIkqQOYnEmSJElSBzA563bLhkeeL362btNSSf17T/xHBtdcbZwDkyRJkjQaJmfdbvLUkedzZzd1yZTJk9jvxRuNU0CSJEmSWmFy1kseu63ppq/fYeMVz5cuWz4e0UiSJEkaBZOzXjK3+eRsr202WPH8uvufGo9oJEmSJI2CyVkvGcXI2WqTRz71H/zejZbUlyRJkgpmctZL5t0NwwuLjkKSJElSC0zOeklaDo83VxREkiRJUmcxOet2U9eGmUOw9Wuzj8/eH4YXNLysVFb/iH22HucAJUmSJDXD5KxXTHtpS5e9cafpK54/NrSoXdFIkiRJGiWTs17RYnK21YZrr3i+3//8wcIgkiRJUkFMznpFeXK2bLi4OCRJkiS1xOSsVwxuPvL8ibuKi0OSJElSS0zOekXEyPPHbi0uDkmSJEktMTnrRY82n5yVqjZe/sl9Vhzb4biLXXsmSZIkTTCTs1706M2jvmTa4BrjEIgkSZKkZpmc9aLH74QllsWXJEmSuonJWS9avgTm/rXoKCRJkiSNgslZr5pz05i7SCm1IRBJkiRJzTA561W//S+YOQjDC1ruYta985hx9IUWB5EkSZImgMlZr5i6Nswcgned19LlpaqNs084cMWxn974cLuikyRJktSAyVmvmb5L27q6/M4n2taXJEmSpPpMznrNutNg3U3a0tWy5a45kyRJkiaKyVkv2mSnoiOQJEmSNEomZ71ok53b0s36a6+24vkOx11sYRBJkiRpHJmc9aI2rTt7y86btqUfSZIkSY1NKToAjYPy5OzZR2GDbZq+tFS1EeD+ec/x7VkPtDk4SZIkSdU4ctaLVl9n5PnDN7TczcYDa7QhGEmSJEnNMDnrdb/89zFvRl3OtWeSJEnS+DA5kyRJkqQOYHImSZIkSR3A5KwXTV0bPvcMrDOt6EgkSZIkNcnkrFdFwGa7jamLUuXG2Scc2KagJEmSJNVictbLxpic1WNhEEmSJKm9TM562aZlyVlKxcUhSZIkqSGTs142bceR50/9rbg4JEmSJDVkctbLJk8def5I65tRS5IkSRp/Jmf94uE/j0u3C4eXMuPoC11/JkmSJI2RyVm/uPmHMHMQhheM+tJ6VRuXLlvejugkSZKkvmdypjH51c1zig5BkiRJ6gkmZxqTb1xpoRFJkiSpHUzOetnUtWHmELzsX8btFkPPL1nx3L3PJEmSpNaZnPWDTXcvOgJJkiRJDZic9YPNyjajXt76qFa1wiAH7LDRWCKTJEmSlDM56wcbbjvy/LHb29r1kftv27iRJEmSpIZMzvpBlH2aH7iqrV1PG1xjlWOuPZMkSZJGz+Ss39z/x6IjkCRJklTFlKID0AR7cFa2GfWxc7Jqji0orT0DHB2TJEmS2sSRM0mSJEnqACZnkiRJktQBTM40bhYOL2XG0RdaHESSJElqgslZP5i6NswcgqPKyug/+1hbuq6295kkSZKk0TM56ydrrT/yvM0l9au5/v6nxv0ekiRJUq8wOetXf/vDuN/i9MvuXfHcvc8kSZKk+kzO+tUDV0FK43qLux57dlz7lyRJknqJyVm/WvAEHL9etufZ8IKio5EkSZL6nsmZ2qJaYZCNB1YvMCJJkiSpu5icadx8bL9tVjnm2jNJkiSpOpMzjZsDX7Jx0SFIkiRJXcPkrJ+U9jv73DOwzrRxv11EjPs9JEmSpF5hctaPImCr1xQagtMbJUmSpJVNKToAFWSr18BtP2l7t6XCIICJlyRJkjQKjpz1qxmvHnm+YF5xcUiSJEkCTM761zr/MPL89Je535kkSZJUsMKTs4j4SETcHxGLIuLGiHh1g/brRcSZEfFofs0dEXFQ2fljIuLPEfFsRDweEb+KiO0q+rgyIlLF47zxeo39qtreZ5UWDi9lxtEXuv5MkiRJfa/Q5Cwi3gGcBpwI7AJcBVwUEVvUaD8VuBSYAbwd2A74IPBIWbN9gDOBPYADyNbVXRIRa1d09y1gk7LHv7flRUmSJElSC4ouCPIJ4JyU0tn5x0dFxIHAEcAxVdq/H1gf2CultCQ/9mB5g5TSG8o/joj3AY8DuwJ/LDu1MKX02NhfgsbirseeLToESZIkqSMUNnKWj4LtClxSceoSYK8al70JuBY4MyLmRsTtEXFsREyuc6vB/N+nKo6/OyLmRcRfI+IrEbFug3hXj4iB0gOo217NOfaXt694bnl9SZIk9bMipzVuCEwG5lYcnwvU2iF5a7LpjJOBg4AvAJ8EPlOtcWS7IJ8CXJ1Sur3s1A+AdwH7Ap8H3gb8okG8xwBDZY+HG7TvbKUNqff77Ljfqt7aswefXDju95ckSZK6QdHTGgFSxcdR5VjJJLIpih9KKS0DboyI6cCngBOqtD8DeBmw90o3TOlbZR/eHhH3ADdExMtTSjfVuPfJZIleybp0e4IGK29GvXRRlrRNoEkBy2t9tiVJkqQ+UuTI2TxgGauOkm3EqqNpJY8Cd+eJWckdwLR8muQKEfE1smmQr00pNUqibgKWANvWapBSWpxSml96AL2xWGqjl4w8//LWE15S//BXb7XKMac3SpIkqR8VlpyllIaBG8kqKpY7ALimxmWzgG0iojzuFwGP5v0RmTOAfwL2Synd30Q4LwFWI0v++kvEhN2q2vTGD+y9anImSZIk9aOi9zk7BTg8It4fEdtHxKnAFsBZABHxvYg4uaz9N4ANgNMj4kURcTBwLFnp/JIzgfcAhwLPRsS0/LFm3ucLI+K4iNgtImbke6T9FPgLWfKnCTR50sQlh5IkSVInK3TNWUrpxxGxAXAc2V5jtwMHpZRK5fG3AJaXtX8oIl4PnArcSra/2enAl8q6PSL/98qK270POBcYBvYHjgTWAR4CLgSOr5guqYLtcNzFAMw+4UDWmtoJyyMlSZKk8VP4b7wppa8DX69xbt8qx64l22C6Vn91h2JSSg+RbVStgpSmNwKuK5MkSZJyRU9rVCdKlk+UJEmSJprJmVY197aiI1jJwuGlzDj6Qis4SpIkqaeZnPW70mbUx84ZOXbX7ybs9vU2qJYkSZL6icmZVjXrtAnf76xZ7oEmSZKkXmVypo53wa39t/2cJEmS+o/JmTpCvemNJ//2zgIikiRJkiaWyZk63uKly1c55vRGSZIk9RqTM9X23NyiIwBg+uAaRYcgSZIkjTuTM9V298VFRwDAl//5ZTXPOYImSZKkXmFypkzVkvoXTXgY1daevXjauhMehyRJkjTRTM5U24Oz4Plnio5CkiRJ6gsmZ6pt+VL40pYdu+dZuYXDS5lx9IVOcZQkSVLXmlJ0AFI1pemNgMmWJEmS+oIjZ+o5FgmRJElSNzI5U23rblJ0BED9DapL/vS3JycwIkmSJKn9TM5U2zYHFB1B0z7109tWOeYImiRJkrqJa860slJJfYC7L4G/fC97vryzE5znlywrOgRJkiRpTCKlVHQMXSkiBoChoaEhBgYGig5nfCwagi9usfKxY+dkCVyBFg4vZYfjVt4ge8dNB7j9kfl1r5t9woGsNdW/R0j8ZgZxAAAgAElEQVSSJKm95s+fz+DgIMBgSqn+L6V1OK1RtU3qnkTm9HfuXHQIkiRJ0piYnKknDK65WsM27oUmSZKkTmZypq7TTPVGSZIkqduYnKkvWclRkiRJnaZ7FhVp4pUqN970ffjNR4uOZhWlETTAJEuSJEldz5EzNbbN/iPPH7+juDjG6Oc3PrzKMUfQJEmS1ClMztTYGoMjz8/eH2YOwvCC4uKpopl1aCdfdNcERiRJkiSNjsmZhCNokiRJKp7JmXpKvRG0w/eeMfEBSZIkSU0yOVPf+PC+L2zYxr3QJEmSVBSTM7VmycKiI6jLvdAkSZLUbUzO1FippP4xj4wcu/vi4uKZIK5DkyRJ0kQyOVPzIkae//o/OrJqoyRJktSt3IRaPW20G1Wff8ucVY7tcFw2Sjj7hANZa6rfMpIkSRofjpxJZY4/v/Ym205zlCRJ0ngyOVPfsEiIJEmSOpnJmcZm3j1FR9BWJ771JQ3bWG5fkiRJ48HkTM0rVW08tmxd1m0/KS6eFtUbQTvwJdNG1ZdTHSVJktQuJmcam5t/lFVt7PPKjSZpkiRJGiuTM43N808VHcGEW55S0SFIkiSpB5mcqW+1WiDkYz+8ueY5R9AkSZLUKjdt0tjEJEjLi45iTEa7F9p19zceLVw4vNT90SRJkjQqjpxpbF64X9ERTLgXbbzOqNo7miZJkqRmmJxp9EpVG2cOwa6HjRxfuqiwkNqlmamO575v95b6NkmTJElSPSZnGput9hl5/uWte6ZqY70kbeoUv20kSZLUfv6WqbGZNLnoCLqOm1hLkiSpGpMzqY5WKzpedsfjTbVzqqMkSZJKTM6kcfD/fn7bqNqbpEmSJMn63mq/Li+tX81oy+2vPmUSi5f23vsgSZKk8ePImcamVLnxU/eOHHvw2uLimQDNTHX8+RF7ttS3I2iSJEn9y+RM7bHaWiPPf/jPPVO1sVXTBteoee6CWx5teL1FQyRJkvqPyZnUolaLhcw8f/ao2juaJkmS1B9MzqQJtvbqrW0/YJImSZLU20zOpDEa7QjaLz+y15juZ5ImSZLUm6zWqPYoFQZ5+kE4/WXZscfvgM12KzauCdRsRcf1157alvstHF7KDsddDMDsEw5kral+O0uSJHUzR87UXmtvOPL81h8XF0eXu/ivj42qvaNpkiRJ3c/kTOPntp9mVRv7sHJjq8VCSj7zy7+2dJ1JmiRJUvcyOdP4ef7poiPoWgNrjG2KokmaJElS9zE5k8ZRqyNo53/sVW25v/ulSZIkdQ8rCEgToNliISVrr974WzOlNKoYSsVDSiwiIkmS1FkcOVN7lao2zhyCzV5RdDQdaazr0Ure9a3rx3S9Ux8lSZI6i8mZxs8u7xl5ftL0viwMMp7uffy5okOQJElSGzmnSeNnhzfB+f9ZdBQdqzSCVr5f2Wh89LUv5Iwr7htzHNXu75RHSZKkiefImcbP5PZsttzrWp3meNirZjRss2jJspZicsqjJEnSxPNP41KHGG3RkGYcdPrVY7q+NKLmSJokSdL4c+RM6kDtKhoyf1F7kjxL8kuSJI0//xSu8VOq3PjkvfC1XbNjJ03P/j12TnZedY11Xdqp79iJj//4lrbG5Po0SZKk8eHImcbfupsUHUHXa3Uk7dXbbtiwTavr0kpcnyZJktQe/rlb6iLjsS7t4K/Oaks/5aN7jqZJkiSNnr89SV1qrFMeS4aeX9LGqDJOfZQkSRo9pzWqGG5K3TZjLR7yxbft2OaIVuXUR0mSpMZMzjT+SoVBPnlX0ZH0tFaTtNdtv3HDNif99s5Ww1qJVR8lSZJqc56RJs7q6xYdgVr0i5seaXufTn2UJElamb8JST1mPIqG7PXCDbjmvifb0lct1dbOmbBJkqR+4rRGFWvxgmztmevPxkW7NrP+6rt2btjm8jsfH9M9qnGtmiRJ6if+SVrF+vs1RUfQF9pV2bGeT//stnHpF0amQN7w2f3Z7QuXrXTO0TVJktQrOmLkLCI+EhH3R8SiiLgxIl7doP16EXFmRDyaX3NHRBw0mj4jYvWI+FpEzIuIBRHxm4jYbDxen+q45mtFR9BX2jWSVs0mg2u0vc9mOLomSZJ6ReHJWUS8AzgNOBHYBbgKuCgitqjRfipwKTADeDuwHfBB4JGyNs30eRrwVuCdwN7AOsAFETG5jS9P5UpVG4+dM3LskRuKi6ePlZK0B754cNtGnX7z0b0atrn63nltuVc1JmmSJKnbFZ6cAZ8AzkkpnZ1SuiOldBTwEHBEjfbvB9YH3pJSmpVSejCldHVK6ZZm+4yIQeADwCdTSr9PKf0FeA/wUuB11W6aj7QNlB6ApQfVE9o1mhYRDdscdd4tDduMleX6JUlStyp0oUY+CrYr8MWKU5cAtf4M/ybgWuDMiHgz8ATwQ+BLKaVlTfa5K7BafgyAlNKciLg9b1NtUc4xwOeafGlq1qTVYPmS7PlJ07N/j52TjbJpQpVXeYTq1RPHauOB1Zk7f3HVc3/5+9NtvResWq6/xHVqkiSpExU9crYhMBmYW3F8LjCtxjVbk01nnAwcBHwB+CTwmVH0OQ0YTilV/jZY774nA4NlD9entcOObys6AtUwHlMfL/jYq2qe++D3bmrLPZrh6JokSepEnfKn41TxcVQ5VjIJeBz4UEppGXBjREwHPgWc0GKfDduklBYDK/7k38wULjXhFR+CW88rOgo10K5qj/W+bwbXnMLQ8xOfKLm3miRJ6hRFj5zNA5ax6mjVRqw68lXyKHB3npiV3AFMy6c0NtPnY8DUiHjBKO6rdikVBpk5BBu9uOhoNArjWe3x4o/XLdIKwBmX39v2+1aysIgkSSpKoX8eTikNR8SNwAHAL8tOHQD8usZls4BDI2JSSml5fuxFwKMppWGAJvq8EViSH/tJfs0mwI7Ap8f6uqReV7k+DRhzMjNlUuO/FZ17zYNjusdoVBsldFRNkiSNp6JHzgBOAQ6PiPdHxPYRcSqwBXAWQER8LyJOLmv/DWAD4PSIeFFEHAwcC5zZbJ8ppSHgHOB/ImL/iNgF+D/gNuD34/pq1dhJ02HmIAwvKDoSjcJ4jqqVvGmnTcat72Y4qiZJksZT4X8CTin9OCI2AI4DNgFuBw5KKZX+RL4FsLys/UMR8XrgVOBWsv3NTge+NIo+AT4OLCUbOVsTuAw4rGK6pKRRatf6tGqOO2QHfnPLo3XbvPvs69t6z2pKr8uRNEmS1E4d8VtFSunrwNdrnNu3yrFrgT1a7TM/vwj4WP5QUUrrz4YXjJTSV08YzyStnrsee3bC7uXUR0mS1E7+BiFpXFWuTxvvhO2Yf9yOky+6q26bf//+jeNyb7D6oyRJap2/MUiacOM5qva2XTdrmJzd+OAzbb1nPc28RhM4SZIEJmfqZKVpjsfOyaY/qudM9KhayecO2Z7jz7+jbpsfXvf3cY2hnNMjJUkSdEa1RklaYSKqPh6yU+P1jadces+43b8ZpcqQ855bxIyjL7RKpCRJfcA/y6ozWBhEFYoqKFKy2QvW5OGnn5/w+9bj6JokSb3NkTNJHa2UpD3wxYMnNBH5+RF7Nmyz73//YQIiqc191yRJ6i3+yVWdb/Fzrj8TsOoatfEcVZs8KRq2eW5xZyRF9d4HR9ckSeoejpyp891zadERqENNxPq0en764brbLQLw+lP/OAGR1ObomiRJ3cM/p6rzXX1K0RGow5WPqE1kErLVho1HcZ9asKRhm9N+P/7FRxxdkySp8/m/sTpLtcIgT91XbEzqKpVTH0uKKizy/Q/szr+e8+e6bf7vT7XL9t/80PjvyVZ6X2747P7s9oXLABM2SZKK4P+86i6uPVOLitpTbftNBhq2+dc9tuD7NRK0w797Y7tDakqt98UETpKk8eP/qup862wMz80tOgr1oIksMFLPka/btmZytvG6qzP32cUTHFFzLO0vSVJ7+b+oOt+rjoSLjy06CvWBotau1XPhkXuvGKmq5dM/u3WCoqmv2vTIEhM3SZIa839KdabS2jOARfNNzjThOm3tWj2X3/lEwzb3Pv7cBERSW+V75vRISZJW5f+G6nyTJhcdgbRCUWvX6vnwPltz1h/+VrfNO7953QRFM3pWkZQkKeP/eupOFgZRByl67drhr96qYXK25mqTeX7JsrptTvrtne0Ma8zqvY/Vpk6WmNRJkrqV/3tJUpt14ujaFZ96DXucdEXdNr+46ZGG/Zx3/UPtCmnc1Fv7VslETpLUSfwfSZImQK01bCXjncBNmTSpYZv3vWoG35n1QN02X7nk7ob9LF5af4SukzTznrs+TpI0UfwfRp2v2sbUUg+qlsBNZNXI/3jtCxsmZwfssBGXzn68bpu9v3hl+4LqMK1MsywxsZMkNeL/Eupurj1TjyslbEVPiyw5+Z9eyqWz6ychqYl+jj9/dnsC6iKjHaUrMamTpP7hT3tJ6gJFFx0ZjUs//moOOPWqum3Ov+XRhv2cfdX97Qqpq41mDZ1TMCWpu/lTW5K6UCdumF3ygrWnNmzzoddsxTf/WD/5qleBct///sOo4+o3Y5mCWZnklfdn0idJ48efruoe9daeLV7gFEf1rUbFRqDzErgPvWbrhsnZW3aezq9unlP13HOLG7+eV5xYPwEBSKmZSZj9rTLJqzeSZ+ImSWPjT1D1hrt+W3QEUkerlcB18vTIz75x+5rJ2c8+vAdvP+tPda9f3kTetfeXrmzY5pmFSxp3JGDVRM4ETpJGx5+O6g1XfaXoCKSuVG1PtpJOTtxmbNh4dPx3R+7NG06/um6bxUuXN+zndaf8sWGbOc8837CNMs0kcJVcSyepX/jTTb1hqPM3xpW6Ta3Rtk6qHlnPhuuu3rDNr/5jL95y5jVjvtebzmjcx3u//eeGbRYt6Z494ori6JykXuZPLnUf9z2TClf0ptrtstkL1mzY5rpj9+OVJ11et83qUyY1HIX765z5De9Vb5rlCX24/UCr2jE6V60fkz5J482fMOoN628NT+WV3SwMInWEeglcpxUoqWfypGjYZtbRr234y/9/v/2lfOpnt7Ucx2+a2H7g9ac2noJ5xuX3Nmxz99xnm4qpV9X6o0Kre9WVmNxJasSfEOoNr/kU/OqIoqOQ1KRu2retXV774o0atvnjp/fhNV+uvk3AEftuzTeurL29AMBTCxoXLzn3mgcbtjn0W9c3bHPxXx9r2EYrG82edSZyUn/yu17dqzS9EWDxcyZnUhert76tlm6ZOjka9X4Z/8DeWzVMzn74wVc0TKwOfeXm/PC6+ut011trtYZVKj/zy7/WPQ/wnz+6uWEbVTfWUbrRTuWcyDYmnlJtfmeoN0SVaUdOb5R6Xj+OwNXzoo3XbdjmEwe8qGFy9vtPvKbhL9u7brkeNz74TN0219z3ZMN4/uMHf2nY5qYHn27YRt2jXRukVyolffV+DpgkqtP5FSlJ6hn1tgaoppvWvnWa//3XXRv+Iv25Q7bn+PPvqNvmuvufanivD33/pprnPnDuDQ2vf+DJBQ3bqPuVTxsdTftKEzmiWJkgVkssm0kim/nDlMlod/AzJEnqW924OXc3OWSn6Q2Ts5lv2oGZv6lfiXKL9dfi708trHruloeHGsbx9m/U37Ac4NRL72nY5vI7H2/YRhqNZn7WNDvFdSz36uRpsM206aXEszdehSRJbdTMCJwJXHu88WWbNEzOfvGRPWv+ovalt72U//fz+lUw11l9Cs8trj9K+oPr/l4/UODTTVTbfN93Gu9n9/vZcxu2kdSfTM7UG+rtfebaM0njYLRTKCvbmNy1x/7bN66CeeWn9mn4V/h/23NLvndt/UqWO2022HCk7rZHGu9nd/Qvbm/Y5uM/vqVhmwvqbK9w2R2NR/nOv2VOwzbnznqgYZur75nXsI2k5picSZJUACtUdpb/3H+bhsnZOYft1pb97JpJ8q5qIuGZWWdj8kajiUDDKacAZ1xxX8M2RzWRSL7jfxtPLf1zE+sPmykyU88Tzy4e0/XSeDM5kySpi9Tb3BtGP4JXjYVSWtfMfnbNJHmfOejFnPjbO+u22XPr9bn2b9UTmp03H+Tmh+ongHu9cIOGyc4hL9uE82+tvwH69pusyx2P1t+4/L4nGhdlOaKJyp3NbM+w95euqHnuH0+/uuH1bz3zmoZt3nxG4zb/clbjhPSI/6td7KbkmF80TrQbTQ0GOPK82u/dN65snITf2sT6zqXLljdso/pMztRb6k1vlCQ1pVECWOI0zfHz1pdv2jA5+9qhu9RM8s5+b+ME8Kvv2rlxxc037dAwOfv+B17RsJ8zDt2Zj/6wfmL1wn9Yu2ESt920dbnrsfqJ4KIltROEyREsS6nu9Q89/Xzd8wCPPNO4zd/mNU5I//xA420iLp3deIrqBQ0+RwCz7q2diJ9z9QMNr39/E5VR9zi5dmJccsjXZjVs82/n1N+vEeCLF9X//uhWk4oOQJowQ3Ng5mD2GLassiS1UymhK39suM4aqxyrfPRKhTXVt8fWGzRs8+N/36Nhmx8c/oqGbX7z0b1qnrvuM/s1vP6c9+7asM23D9utYZuz3rNLwzYnvHmHhm3+6/UvatjmY/tt07DN5w7Zvua5f9lts4bXb7remg3bNOPRoUUN28xuMBIL8LMbH2lHOB3Hn4jqH9ecXnQEkqQKrYzSlXMKpipNH2MSsdPm6zVs87LNBhu22W3G+g3bHPTSTTju1/WnJL7zFZvzlUvurtvmvXttydcuv7dum3pbW3z6Ddvxkxsernv9rz+6V8MR0ss/+Rr2+58/1m1z7vt247Dv1B+FO+0dOzVcy3j43jM4u4kRv25jcqb+cfMPio5AktRmY0nuGk3FnH3CgUDtzYolrWxgzdUattlx08aJ7d7bbtiwzYf3faHJmdQ1qq09S2Vz0C2vL0l9r9ntEMayZUKJa/IkNcPkTP1jtbVgycKio5Ak9aGx7ovXSptaWy/MPuHAFWv92hmHCag0diZn6h97/Adc9d9FRyFJ0oRpdtrnRNyr00YUy5PUWnGUtylXvtaxVhuYmCS81Tblr7mZrxPXd06MSA3Kiaq6iBgAhoaGhhgYGCg6HNVSPq3x43fAqRWVipzWKEmS1FFqjfjCyBrQGz67/4oCJfUS5Ikyf/58BgcHAQZTSvNb7cfkrEUmZ12o3t5nJmmSJElqUbuSM/c5kyRJkqQOYHImSZIkSR3AaY0tclpjF3N6oyRJktrIaY2SJEmS1ENMziRJkiSpA5icSZIkSVIHMDlT/5m6NswcytaXVbrzIpg5mD2GF0x8bJIkSepbJmdSud8fV3QEkiRJ6lMmZ+pf1UbQ5j9SXDySJEnqayZnUi0nTXd6oyRJkiaMyZlUbtsDi45AkiRJfcrkTCq3v2vOJEmSVIwpRQcgFa609gyqT2E8aXr277FzsraSJEnSOHDkTJIkSZI6gMmZJEmSJHUAkzOpWVZvlCRJ0jgyOZPKVdv7TJIkSZoAJmeSJEmS1AFMzqTRGl6YTW90iqMkSZLayORMqqbe9Mb7/zDx8UiSJKnnmZxJo3XBx4uOQJIkST0oUkpFx9CVImIAGBoaGmJgYKDocDTehheMbEZdjRtUS5Ik9a358+czODgIMJhSmt9qP46cSaO17rSiI5AkSVIPMjmTRust/1t0BJIkSepBJmfSaG38kqIjkCRJUg+aUnQAUlcoVW+E6uXzS+vRXHsmSZKkFjlyJkmSJEkdoPDkLCI+EhH3R8SiiLgxIl5dp+1hEZGqPNYoa/NAjTZnlrW5ssr588b7tUqSJElSLYVOa4yIdwCnAR8BZgH/DlwUETuklP5e47L5wHblB1JKi8o+3B2YXPbxjsClwE8r+vkWcFzZx8+P+gWoP5WmOFYrr+/0RkmSJLWo6DVnnwDOSSmdnX98VEQcCBwBHFPjmpRSeqxWhymlJ8o/joijgfuAP1Q0XVivH0mSJEmaSIVNa4yIqcCuwCUVpy4B9qpz6ToR8WBEPBwRF0TELg3u8R7g22nV3bbfHRHzIuKvEfGViFi3QbyrR8RA6QHUba8+UBpBO3bOqueGF8LMwexRrYCIJEmSVKHIkbMNyaYfzq04PheotcvvncBhwG3AAHAkMCsidkop3VOl/VuA9YBzK47/ALgfeIxs2uPJwE7AAXXiPQb4XJ3z0ojHbi86AkmSJHWZoqc1AlSOaEWVY1nDlP4E/GlFw4hZwE3Ax4D/rHLJB4CLUkorDW2klL5V9uHtEXEPcENEvDyldFONOE8GTin7eF3g4Rpt1e9++cGiI5AkSVKXKTI5mwcsY9VRso1YdTStqpTS8oj4M7Bt5bmI2BJ4HfBPTXR1E7Ak76dqcpZSWgwsLuu/mRDVrxaULX20SIgkSZKaUNias5TSMHAjq04lPAC4ppk+IsuQdgYerXL6fcDjwIVNdPUSYLUa/Uj1VVt7tu4mxcUjSZKkrlT0PmenAIdHxPsjYvuIOBXYAjgLICK+FxEnlxpHxOci4sCI2DoidgbOIUvOzirvNCImkSVn300pLa0498KIOC4idouIGRFxEFmZ/b+QlfOXxu5t56x67KTpFgiRJElSTYWuOUsp/TgiNiDbb2wT4HbgoJTSg3mTLYDlZZesB3yTbCrkEFlC9ZqU0vUVXb8uv/bbVW47DOxPVkxkHeAhstG141NKy9rxuiQ2fFHREUiSJKnLxKoV5tWMvJz+0NDQEAMDA0WHo05TbYPqkv+6F76yTfbcdWiSJEldb/78+QwODgIMppTmt9pP0dMaJUmSJEmYnEkT7+YfFh2BJEmSOpDJmTQeqlVwLLn8hImPR5IkSR3P5EyaaMvLCohawVGSJEk5kzNpPFUbQdto++LikSRJUscyOZMm2pvPWvWYI2iSJEl9z+RMmmiDmxYdgSRJkjpQoZtQS32jNL0RHB2TJElSVY6cSZ3E6Y2SJEl9y+RMkiRJkjqAyZk00ertgVYyvDAbQXMUTZIkqW+YnEmdaPmyoiOQJEnSBDM5k4pSbwTt8hMmPh5JkiQVyuRM6kQ3/2DkuUVCJEmS+oLJmSRJkiR1AJMzqWjVpje+4kOrtnMETZIkqaeZnEmd6NWfKjoCSZIkTTCTM6kTRRQdgSRJkibYlKIDkJQrTW+E+lMXT5qe/XvsnOwaSZIk9QRHziRJkiSpA4w5OYuIgYh4S0Rs346AJFF/D7SS4YVZgRCLhEiSJPWEUSdnEfGTiPho/nxN4AbgJ8CtEfG2NscnSZIkSX2hlZGz1wBX5c/fCgSwHvCfwGfbFJckqD+C9sSdEx+PJEmSxk0rydkg8FT+/A3Az1NKC4ELgW3bFZikBs47dOS5e6BJkiR1vVaSs4eAPSNibbLk7JL8+AuARe0KTFIDi+cXHYEkSZLaqJXk7DTgB8DDwBzgyvz4a4Db2hOWpJVUm9641T6rtnMETZIkqWuNOjlLKX0d2BN4P7B3Sml5fupvuOZMmjhv+UbREUiSJKmNWtqEOqV0A1mVRiJiMvBS4JqU0tNtjE1SPZOn1j7nRtWSJEldZ9TJWUScBtyWUjonT8z+AOwFLIyIN6aUrmxzjJJKStMbwamLkiRJPaaVNWdvB27Jnx8CbAW8mGwt2oltikuSJEmS+korydmGwGP584OAn6aU7gbOIZveKGki1NsDrWR4YVYgxCIhkiRJHa+V5GwusEM+pfENwO/z42sBy9oVmKQ2mHt70RFIkiSpSa0UBPkO8BPgUSABl+bHXwnc2aa4JLXDL/+96AgkSZLUpEgpjf6iiLcDm5NNaXw4P/Ze4JmU0q/bG2JniogBYGhoaIiBgYGiw5GyaYulKo31WMFRkiSprebPn8/g4CDAYEppfqv9tFpK/2dVjn231SAkjZM1XwDPu8OFJElSN2hlzRkRsU9EnB8R90bEPRHxm4h4dbuDkzQK1QqEvPnrq7Y7aboFQiRJkjrQqJOziHgPWRGQhcBXgTOA54HLIuLQ9oYnaUw2273oCCRJktSkUa85i4g7gG+mlE6tOP4J4IMppe3bGF/Hcs2ZukIz69BcgyZJkjQm7Vpz1sq0xq2B86sc/w3ZhtSSJEmSpFFqJTl7CNi/yvH983OSuokbVUuSJHWEVqo1/g/w1YjYGbiGbK+zvYHDgCPbF5qkMSsVCak3vTEtn9iYJEmSVNWok7OU0jci4jHgk8C/5IfvAN7RL3ucST3lipOKjkCSJEm0uAl11Y4iVgM2SSn9vS0ddjgLgqgruVG1JElS2xVZEKSWHYD729ifJEmSJPWNdiZnkjpdtY2qX/nhVdu5UbUkSdKEMzmT+t3enyw6AkmSJGFyJimi9jlH0CRJkiZM09UaI+JlDZpsN8ZYJE2U0vRGMPGSJEnqEKMppX8z2Z5m1f7MXjrentKPkiRJktRnRpOcbTVuUUgqTjMbVQ8vHDlnmX1JkqRx0XRyllJ6cDwDkdTBZru/vCRJ0nhr2ybU/cZNqNWzqo2gTV4Nli1Z+ZgjaJIkSUBnbkItqVdVJmaSJElqO5MzSSurtlH1tJ1WbWeZfUmSpLYyOZPU2Fv/t+gIJEmSep7JmaTG1t6w9jlH0CRJktpiNKX0AYiIv1B9P7MELALuBc5NKV0xxtgkFcmNqiVJkiZUKyNnvwO2BhYAVwBXAs8BLwT+DGwC/D4i3tymGCVJkiSp54165AzYEPiflNLnyw9GxGeBLVNKr4+I44H/D3BzJKlfuFG1JEnSmLQycvYvwI+qHD8vP0d+frtWg5LUYapVcKx0x/kTF48kSVIPaiU5WwTsVeX4Xvm5Ur+LWw1KUoeql6T99pMjzy0SIkmSNGqtTGv8GnBWROxKtsYsAa8ADgdOytscCPylLRFK6g5pedERSJIkdbVIqVrhxQYXRbwb+CgjUxfvAr6WUvphfn5NIKWUFtXooutFxAAwNDQ0xMDAQNHhSMUYXjCyzmznd8PNP6jezjVokiSph82fP5/BwUGAwZTS/Fb7aWXkjJSGR7oAACAASURBVJTSD4Aav4VBSun5VgOS1KX2n1k7ObNQiCRJUkMtJWcA+bTG7cmmNc5OKTmNUepnEUVHIEmS1NVa2YR6I7LKjPsCzwABDEbEFcA7U0pPtDVCSZ1rtBtVW25fkiSpplaqNX4NGABe8v+3d+dhsl1lvfi/LwknDEl3IBgCQoAf5KIMl2DEAZkUAhiVMFwliA8EEPwh6gUcgxqCCHgxIMogAtFImFVk0kiYL4agJEEkDBIEwpCJEOgOCTknJOv+UdWcSnd1VXWd7q6qrs/nefbT1XuvvWsVmzo537PWendr7eattZsluVt3319sZucAAADmxTjTGh+a5EGttc+s7GitfbqqnpbkjE3rGTBbVkbReouErPatC/a+NoIGAHA944yc3SDJNX32XzPm9YB5cdojJt0DAICpteFS+lX19iQHJ3lMa+3C7r7vT6d64zdba3Pxty+l9GGIQSNo/RhBAwBm1GaV0h9npOvXkhyU5EtV9d9V9fkkX+zu+41xOwLMgXv84qR7AAAwtTa85qy19pUkP1RVRyf5gXSqNX66tfbeze4csMMc/UfJJ97Q/5hKjgDAnBv7OWettfckec/K71V12yTPaa09cTM6Bsy4UQqEAADwPZtZwOPmSR6/idcD5slXP7b39fNvnZy0ONqz0wAAdoiJV1esql+tqi9W1dVVdU5V3XdA2+OrqvXZbtTT5qQ+xy9edZ3qtruwqr5TVR+sqrtu5eeEubUygnbSUrLrJuu3+3uD7gDAfJtoOKuqRyd5SZLnJblnkg8nOb2qDh9w2nKSW/VurbWrV7X51Ko2d191/HeSPDOd4ib3SnJxkvdU1UH79IGA8X33O5PuAQDARE165OyZSU5prb2mtfaZ1trTk3wlyVMHnNNaaxf3bn3afHdVm6+vHKiqSvL0JM9rrb21tXZeOtMxb5Jk3VJyVXVAVS2sbOlUpwQ2YmUU7VkXrj12hwes3Wd6IwAwR0YuCFJVbx3S5OCNvHFV7UpyVJI/WXXojCT3HnDqgVV1QZL9kvxHkj9srX18VZsjqurCJLuT/FuSZ7XWvtA9dockh3XfJ0nSWttdVR/qvu9frfO+JyR59tAPBozn4a9I/uwuk+4FAMDEbGTkbGnIdkGS127gerdIJ2Bdsmr/JemEp34+m+T4JA9L8pgkVyc5s6qO6Gnzb0kel+QhSZ7cvdZHquqQ7vGVa2/kfZPkBUkWe7bbDGgLDNJvBG2/Xeu3N4IGAMyBkUfOWmtP2KI+tFW/V599K334aJKPfq9h1ZlJzk3y6+k+ALu1dnrPKZ+sqrOS/Hc6UxdfPM77dq+7O52RuJX3Xq8pMKqVkJYIXgDA3JvkmrPLklybtaNVh2btqFZfrbXrknwsyRED2lyZ5JM9bVbWqI39vsCE7LmqM4JmFA0A2IEmFs5aa3uSnJPk6FWHjk7ykVGu0S3ucWSSiwa0OSDJD/a0+WI6Ae3onja7ktx/1PcFtsCgYiEAAHNg5GmNW+TFSU6rqrOTnJXkKUkOT/LKJKmq1yb5WmvthO7vz05nWuP5SRbSmcp4ZJKnrVywqk5O8s4kX05nNOwPum3/NumUeqyqlyR5VlWd373Ws5JcleQNW/x5gX1x7t/uff38W3d+PuvCTrADAJhxEw1nrbU3dwt1nJjO88jOS3JMa+2CbpPDk1zXc8rBSV6VzpTEpSQfT3K/1tq/97S5TZI3plNw5OvphLkf67lmkrwwyY2TvCLJzdIpIvLg1toVm/sJgQ1bGUHbc+XeALbi/c+dTJ8AALZBtbZuDQwG6D7rbGlpaSkLCwuT7g7sPP3C2SBG0ACACVleXs7i4mKSLLbWlse9zqQfQg0wuvv/3qR7AACwZYQzYDr1KxByr19ev71noQEAM044AwAAmAKTrtYIMNhGH1S95yqVHAGAmWTkDNhZvvOtSfcAAGAswhkwO0Z5UPUbH719/QEA2ERK6Y9JKX2YAqOW2ze9EQDYQkrpA/RzyBGT7gEAwFiEM2B29Zvm+Jg3rW2nzD4AMAOEM2BnudHi+seENABgiimlD8y+jZbbBwCYQkbOgPmz56rOCJpRNABgighnwM4ySrn9K7+xff0BABiRcAbMn9Metve1dWgAwJQQzoCdadAI2rcv2f7+AAAMIZwB8+cuD1+7zwgaADBhwhmws/UbQfvpP12/vZAGAEyIcAbMn6pJ9wAAYA3POQPmw0afhbbnqs4oWtIZddt1063rGwBAjJwB82iUcvvf3b19/QEAiHAG0N9bn7z3tXVoAMA2EM6A+TVoBO3LH9n+/gAAc004A+jnJoes3WcEDQDYQsIZQL8RtF98y+T6AwDMJeEMoJ+Db7f+MSNoAMAWUEofYIVy+wDABBk5AwAAmALCGUA/ozwLbfcVe1+b6ggA7CPhDGBcpz187T4hDQAYk3AGMMigEbRvXbD9/QEAdizhDGBcd/ypSfcAANhBhDOAUfQbQXv4X63f3vRGAGCDhDOAcVUNb7Pnqk5IE9QAgCE85wxgIzb6LDQAgBEZOQPYLqY6AgADCGcA4xrlWWhnvXz7+gMAzDThDGArnflna/cZQQMA+hDOAPbVoBG0G950+/sDAMwk4QxgKz3u7esfM4IGAPQQzgC20s1uP+keAAAzQjgD2CyjFAjpx7PQAIB4zhnA5tvos9Cuunxr+wMAzAQjZwCT9tqf2/vaOjQAmFvCGcBWGmWq47cv2b7+AABTSzgD2A6DQtpdH7l2nxE0AJg7whnApP30C9c/plgIAMwN4QxgO41b0REA2PFUawSYhFErOl5w5t7Xz7915+ezLuycDwDsKEbOAKbZ3x0/6R4AANukWmuT7sNMqqqFJEtLS0tZWFiYdHeAnWDPlXtHx0ZhBA0ApsLy8nIWFxeTZLG1tjzudYycAUyzn33JpHsAAGwT4QxgWvQrFvIDP7t+e+X2AWBHEc4AAACmgGqNANNm1EqOK/ZcpZIjAOwARs4Aptkoz0X7+uf2vjbVEQBmlnAGMOted+ykewAAbAKl9MeklD4wEcrtA8DUUUofgI6HvGDSPQAANoFwBjBL+q1Bu/vPr9/eGjQAmBnCGcA82HNVJ6QJagAwtZTSB5hFGy23DwBMPSNnALNulHL7n3zL9vUHABiLcAYwD979rL2vrUMDgKmklP6YlNIHpla/cvv77Uqu3dO/vXL7ALBPlNIHoL9+0xwf987J9QcAGIlwBjAPDrnj+sdMcwSAqaBaI8BOtdGKjnuu2jsd0lRHANh2Rs4A6Njz7Un3AADmmoIgY1IQBJhJ/YqFrDjkiOQb519/nxE0ABhqswqCCGdjEs6AmTYopPUjpAHAulRrBGBzHXaP9Y8pGgIAW044A5hH/crtH/eGyfUHAFCtEWCuqegIAFNj4iNnVfWrVfXFqrq6qs6pqvsOaHt8VbU+24162pxQVR+rqiuq6tKqeltV3XnVdT7Y5xpv2srPCbAjXHHx3temOgLApppoOKuqRyd5SZLnJblnkg8nOb2qDh9w2nKSW/VurbWre47fP8nLk/xYkqPTGR08o6pW//Puq1dd51f2+QMBzLJ+Ux1X++sHb19/AGDOTHpa4zOTnNJae03396dX1UOSPDXJCeuc01prF69zLK21h/b+XlVPSHJpkqOS/N+eQ1cNug4AfVxz1dp9pjkCwKaY2MhZVe1KJzCdserQGUnuPeDUA6vqgqr6alW9q6ruOeStFrs/L1+1/7FVdVlVfaqqTq6qg4b094CqWljZkgxsDzCzBo2gHfOi9c8zzREA9skkpzXeIsl+SS5Ztf+SJIetc85nkxyf5GFJHpPk6iRnVtUR/RpXVSV5cZJ/ba2d13Po9d3zH5DkuUkeleStQ/p7QpKlnu2rQ9oDzLZ+Ie0uxw4/b89VnZAmqAHAhkx6WmOSrH4KdvXZ12nY2keTfPR7DavOTHJukl9P8ht9TnlZkv+Z5D6rrvPqnl/Pq6rzk5xdVT/UWjt3nX6+IJ2gt+KgCGgAg5nyCAAjm2Q4uyzJtVk7SnZo1o6m9dVau66qPpZkzchZVb00nRG2+7XWhoWoc5Nc071O33DWWtudZHfP9UfpIsDs22i5/fc/d2v7AwA71MSmNbbW9iQ5J52Kir2OTvKRUa7RnbZ4ZJKLevdV1cuSPDLJT7XWvjjCpe6a5Ia91wGgj1EqOp77t2v3WY8GAENNelrji5OcVlVnJzkryVOSHJ7klUlSVa9N8rXW2gnd35+dzrTG85MspDOV8cgkT+u55suT/GKSY5NcUVUrI3NLrbXvVNUdkzw2yT+nM3p3lyQvSvLxJGdu3UcFmBMHHXb956H1Ms0RANY10XDWWntzVR2S5MR0njV2XpJjWmsXdJscnuS6nlMOTvKqdKZCLqUTqO7XWvv3njZP7f784Kq3e0KSU5PsSfLAJP87yYFJvpLkn5I8p7V27aZ8MICdbmUEbc+VewPXiuNPT146pJDunqsENQBYZdIjZ2mtvSLJK9Y59oBVvz8jyTOGXG/gYrDW2lfSeVA1APuqX0g7wJNGAGAcEw9nAOwAGy0acl3PRAUjaACQZLLPOQNgXr3uEWv3KRoCwJyr1vo+UowhqmohydLS0lIWFhYm3R2A6dNvPdpGGEkDYEYsLy9ncXExSRZba8vjXsfIGQBbY1DZ/Xs8Zvj5e67qjKQZTQNgTghnAGy/oz2oGgBWE84A2FqjPLh6GOvRAJgDqjUCsD02WtHxSx/e2v4AwJRREGRMCoIAbIJxi4YoFgLAFFEQBICd7QY3XP+YYiEA7EDCGQCTM2g92hP/Zfv7AwATJJwBMJ0Ovt1o7RQLAWCHEM4AmLyNVnR8z4lr9wlpAMw44QyA2fOJN0y6BwCw6ZTSB2B6jFpu/8BbJt++pP+xleqPKjoCMGOMnAEwnQZNdXz8u4afr6IjADNGOANg9tz4Zhtrbz0aADNAOANgum20WMiKq8d+BigATES11ibdh5lUVQtJlpaWlrKwsDDp7gDMlz1X7l1btp4bHZxc/a3+x6xHA2ATLS8vZ3FxMUkWW2tj/+ugkTMAdqb1gllimiMAU0k4A2D2jDLV8SHP377+AMAmEM4AmF2DQtrdf2H4+So6AjBFhDMAZt+4RUN6meoIwIQJZwDMrwvOnHQPAOB7VGsck2qNAFNulIqOg6joCMCINqtao3A2JuEMYEYMCml1g6RdN/h8IQ2AIZTSB4BRDFqP9kv/OPx8RUMA2CbCGQDz65Z3nXQPAOB7hDMA5sO4FR17p/+r6AjAFtp/0h0AgG21EtKS0ULWmx6zdt/KGjbr0QDYRAqCjElBEIAdRGVHAPaBgiAAsB2OeMikewDAnBDOAGDQerRjXz78fBUdAdgEwhkArBi3aAgAbALhDAD21be+sve1io4AjElBkDEpCAIwJ0YpFnLDmyTXXNX/mGIhADveZhUEEc7GJJwBzBkVHQFYh2qNADAtfvL3J90DAHYA4QwARjGoWMhRTxh+voqOAAwhnAHARoxb0bF3GYGiIQD0sf+kOwAAM2klpCWjhay/e/zW9geAmWfkDAC2w5c/snafETQAeghnALCvRpnqeKej1z8mpAEQ4QwANs+gkPbwvxx+vqIhAHNNOAOAzTZu0RAA5ppwBgDT4vIv7H1tqiPA3FGtEQC2ykYrOp76M1vbHwCmWrXe564wsqpaSLK0tLSUhYWFSXcHgFmx58rOqNhGPevCTtgDYOosLy9ncXExSRZba8vjXse0RgDYToPWox37ivXPUywEYMcTzgBgWhzx4PWP7fn23tfWowHsSMIZAEzCRis6vvon1+4T0gB2FAVBAGCSRi0a8p1vbk9/AJgYI2cAMAuOedH6x4ygAewIwhkATItBUx3vcuzw8xUNAZhpwhkATJuNrkdbcc3Ve18bTQOYOZ5zNibPOQNgW43yfLSDbpVccVH/Y56TBrBlPOcMAObJKKNp6wUzAGaCcAYAs2RQSLvf76x/nmmOAFNPOAOAWdQvpP3IU4afp2gIwNQSzgAAAKaAh1ADwCwb9SHWK979rL2vVwqMKBYCMBVUaxyTao0ATK1RKjv2I6QBjEW1RgCgv0FFQxZus/39AWAkwhkA7FT9Qtrx71q/vYqOABMlnAHAPNl14PA2KjoCTISCIACw0220aMiH/s/e14qGAGwbBUHGpCAIADNN0RCATaMgCAAwvkFFQ2551/XPsy4NYMsIZwAwz/qFtF962+T6AzDHhDMA4PqqhrdRNARg0ykIAgBsvGjIp3tG1xQNAdgUCoKMSUEQAHY8RUMARrJZBUGEszEJZwDMjX4h7YCDkt1XDD5PSAPmhGqNAMD26Fc05Jc/MLn+AOxQwhkAsHE3Pnh4G0VDADZkKsJZVf1qVX2xqq6uqnOq6r4D2h5fVa3PdqONXLOqDqiql1bVZVV1ZVW9o6pus1WfEQBm3soI2klLya6bDG9/4bl7X3s+GsBQEw9nVfXoJC9J8rwk90zy4SSnV9XhA05bTnKr3q21dvUGr/mSJI9IclyS+yQ5MMm7qmq/TfpoALBzDXqI9Yo3PmbtPiENYF0TD2dJnpnklNbaa1prn2mtPT3JV5I8dcA5rbV2ce+2kWtW1WKSJyX5zdbae1trH0/yS0nunuRB/d6wO9K2sLIlOWhfPjQA7Hjt2kn3AGCmTDScVdWuJEclOWPVoTOS3HvAqQdW1QVV9dWqeldV3XOD1zwqyQ1727TWLkxy3oD3PSHJUs/21QH9A4D5MGgE7aEvXP88I2gAa0x65OwWSfZLcsmq/ZckOWydcz6b5PgkD0vymCRXJzmzqo7YwDUPS7KntfbNDbzvC5Is9mzWpwHAin4h7W6PHH5eb9EQxUOAObf/pDvQtfpha9VnX6dhax9N8tHvNaw6M8m5SX49yW+Mc80R33d3kt097zvkUgAwh1ZCWiJkAWzQpEfOLktybdaOVh2atSNffbXWrkvysSQrI2ejXPPiJLuq6mbjvi8AsAn+/glr95nyCMypiYaz1tqeJOckOXrVoaOTfGSUa1RnCOvIJBdt4JrnJLmmt01V3SrJ3UZ9XwBgiFEqOn7pw+sfE9KAOTMN0xpfnOS0qjo7yVlJnpLk8CSvTJKqem2Sr7XWTuj+/ux0pjWen2QhnamMRyZ52qjXbK0tVdUpSV5UVd9IcnmSk5N8Msl7t/TTAsC8WQlpe67sBK5e9/jF5BNvGHz+nqv2nvesCzvXA9iBJh7OWmtvrqpDkpyYzjPLzktyTGvtgm6Tw5Nc13PKwUlelc60xaUkH09yv9bav2/gmknyjCTfTfKWJDdO8r4kx7em7i8AbJuj/2h4OOslpAE7WLU2rEYG/XSfdba0tLSUhYWFSXcHAGZH7wjab30+OflOg9s/+I+TM/7g+vuEM2CKLC8vZ3FxMUkWW2vL415HOBuTcAYAm6DfVMeNENKAKbBZ4WzS1RoBgHk2StGQGy2uf6z3OWkKhwAzTjgDAKbbk943WjvVHYEZJ5wBAJM3aATtxgevf97Zp2xdnwC2mXAGAEyPUaY59vrgC9buM4IGzKiJl9IHAFhjJaQlg0PWTb8vufLr/Y8puw/MGCNnAMB0GzSa9qT3DD9f0RBgRghnAMBs6BfSdh24sWuY8ghMMdMaAYDZMuqUxxXLI65fA5gwI2cAwM7210ev3WcEDZhC1VqbdB9mUlUtJFlaWlrKwsLCpLsDAPNtz5V7C4CMQ9EQYB8sLy9ncXExSRZba8vjXsfIGQAw+wYVDTn2FdvfH4AxCGcAwM52xIOHt1HREZgCwhkAsHNs9CHWKz7593tfW48GTIg1Z2Oy5gwAZoT1aMAW26w1Z8LZmIQzAJgxg0La/gck3909+HwhDViHgiAAABsxaMrj4961/f0BWEU4AwDmS7+QdvM7DD9P0RBgi+0/6Q4AAEzESkhLRgtbX/+vva9Xpkea6ghsImvOxmTNGQDsQIPWpdUNknZd/2NCGsw1a84AALbTesEMYJMIZwAAKwYVDXnMm9c/z7PRgE0gnAEArNYvpH3/UcPPUzQE2AcKggAArGejRUO++aW9rxUNATbIyBkAwGb5m4eu3WfKIzAi1RrHpFojAMy5QZUdBzGSBjuOao0AANPmuDcOb2NdGrAO4QwAYBz9iobc5l4bu4Ypj0APBUEAAPbFRouGnHPq2n2KhwAxcgYAsL0+8MeT7gEwpRQEGZOCIADAugYVC7nJIclV3xh8vhE0mCkKggAATKt+69FWPOm9w89XNATmknAGALBV+oW0Aw7a2DUUDYG5oSAIAMBW22jRkLc/be0+RUNgxzNyBgAwbc5/9/rHjKTBjiWcAQBsp0Hr0Vb84MO2rz/A1BDOAAAmYVBI+5kXDz9f0RDYcYQzAIBJGmUkrZ+vnbv3tamOsCN4ztmYPOcMANgyg56TNgpFQ2Bbec4ZAMBONcpoWg34a5wpjzCThDMAgGk1KKQd/8+jXcOUR5gZwhkAwLTrF9IOudP67T/2mrX7hDSYetacjcmaMwBgoqxLg6lhzRkAAP0dfLvhbaxLg6kjnAEAzKJB69Ge+O7t7w+wz/afdAcAANgHKyGtd5rjDUb4K96ln9n7evX0SFMeYSKEMwCAnWAlpCWjTVN87c9tbX+ADVMQZEwKggAAU29Q0ZDaL2nXDj7fCBqMREEQAAAGG7Qu7ckfGH6+oiGwrYQzAICdrl9IW9hgGX7PSYMtZ80ZAMC82Oi6tLc/be2+lWmSpjzCpjNyBgBAf+cPKMlvJA02nXAGADCPBq1HW3HEQ4Zfx7o02DTCGQDAPBsU0o59+cauZTQN9olwBgDAaCNp/Sz3aS+kwVgUBAEAYK+NFg151f22tj8wRzyEekweQg0AzI1BD7NOJRny90mVHdnhPIQaAIDJ+///dXgbRUNgJMIZAACDDVqPduAtN3Yt69FgXcIZAACjGbdoyLuesTX9gR3GmrMxWXMGAJAh69E2wLo0Zpg1ZwAATN4oo2l3uP/29QdmmHAGAMC+GxTSHnXK8PMVDQHhDACATTTuurQ9V+19rWgIc8qaszFZcwYAMKJR1qXd+GbJd77Z/5j1aEy5zVpzJpyNSTgDANigfS0eIqQxpRQEAQBgtgya8njMi4afb10aO5xwBgDA5N3l2I21ty6NHUg4AwBge41bNOT9z92a/sCUsOZsTNacAQBsIg+zZoZZcwYAwM4xymjaze+4ff2BCRDOAACYHoNC2uPfOfx8RUOYYcIZAADTp19I22/X8POuvWbva0VDmDFTEc6q6ler6otVdXVVnVNV9x3xvOOqqlXV21btb+tsv93T5kt9jv/JZn82AAD2wUpIO2kp2XWT4e1f94i1+4Q0ZsTEw1lVPTrJS5I8L8k9k3w4yelVdfiQ826X5ORu+9VutWp7YpKW5B9WtTtxVbs/HvuDAAAweV//7PrHTHlkyk08nCV5ZpJTWmuvaa19prX29CRfSfLU9U6oqv2SvD7Js5N8YfXx1trFvVuSY5N8oLW2uu0Vq9p+e9M+FQAAm2uUoiF3efho1zKaxhSaaDirql1JjkpyxqpDZyS594BTT0zy9dbaKSO8xy2T/EySfm1/t6q+UVX/UVW/3+3Petc5oKoWVrYkBw17bwAAtsCgkHbMyeuft9ynvZDGFNl/wu9/iyT7Jblk1f5LkhzW74Sq+okkT0py5Ijv8fgkVyR566r9f57k3CTfTPIjSV6Q5A5Jfnmd65yQzkgdAADTYCWkjfqMtNc8cP1jK+d7ThoTNOlwtmL1k7Crz75U1UFJXpfkya21y0a89hOTvL61dvX13rC1P+v59T+r6ptJ/r6qfre19o0+13lBkhf3/H5Qkq+O2AcAALbKSkhLBo+AXXfN+sdgCkx6zdllSa7N2lGyQ7N2NC1J7pjk9kneWVXfrarvJnlckod1f7/ekwm7VR/vnOQ1I/Tlo92fd+p3sLW2u7W2vLKlMxoHAMA0GTTl8dFvGH6+oiFM0ETDWWttT5Jzkhy96tDRST7S55TPJrl7OlMaV7Z3JPlA9/VXVrV/UpJzWmufGKE79+z+vGikzgMAMFtu+yMba289GttsGqY1vjjJaVV1dpKzkjwlyeFJXpkkVfXaJF9rrZ3QnZp4Xu/JVfWtJGmtrd6/kOTnk/zm6jesqh9P8mPphLqlJPdK8mdJ3tFa+/KmfjoAALbfRtejrXjfc9busx6NbTLxcNZae3NVHZK9zxw7L8kxrbULuk0OT3LdGJc+Lp21a2/sc2x3kkenU+DjgCQXJHl1kheO8T4AAEyrUdejrfj4aesfE9LYYtXamrobjKA7Mre0tLSUhYWFSXcHAIBRDRpN+/4fTr529uDzf+vzycndMgWCGkmWl5ezuLiYJIvd+hRjmXRBEAAA2F6DioYc12/S1QDWpbGJhDMAAOZTv5BWNfy8b1+6dX1irpnWOCbTGgEAdqBRCojsf0Dy3d39j5nmOJdMawQAgElYL5glpjmyT4QzAABYMWg92opH/fXw63iYNWMQzgAAYLVBIe0O99vYtYymMSLhDAAA1jPKSFo///aXa/cJaQyhIMiYFAQBAJhToxQNGYXiITuGgiAAADAJo4ymLd52+HWsS2MV4QwAAMYxKKQ98d0bu5Ypj0Q4AwCAfdMvpO23a/h5F5y5dp+QNtesORuTNWcAAKzLurS5Ys0ZAABMq1HWpY0yumZd2lwRzgAAYKsMCmm//P6NXcuUxx1POAMAgK3WL6QddNjw8665eu0+IW3HsuZsTNacAQCwT0ZZl3aTWyRXXTa4jXVpE2fNGQAA7HTDghk7inAGAACTMErRkIe+cPh1FA3ZMYQzAACYpEEh7W6P3Ni1rEebacIZAABMg1FG0vp570lr9wlpM0lBkDEpCAIAwJbzMOuZoCAIAADsdKOMpt38jsOvY13aTBDOAABg2g0KaY9/58auZcrj1BLOAABglu23a3ib3Ves3SekTR3hDAAAZsW4RUNedf/1j5nygiXUgAAAEj1JREFUODX2n3QHAACADVoJaclogWr3iDUqVoqPKCAyEUbOAABglo0ymvZzf7F9/WFswhkAAOwEg0LanY9Z/7wzfn/tPuvRJkI4AwCAnWSj69L+883rHxPStpU1ZwAAsBONui7tiIck57978LX2XLX2YdjWpW06I2cAADDPjn35eOcZVdt0whkAAOx045bgX/HWJ69/TEjbNMIZAADMi3FD2hc+sDX94XqEMwAAmDcrIe2kpWTXTYa3v+fjhrfxMOt9JpwBAMA8G2U07YEnDr9Oa3tfm+o4FtUaAQCA61d3TDrBanWFxkHe8L/W7ls5X2XHkQhnAADAWqOW4l9x0SfWPyakjcS0RgAAYLBRpj4e+djh17EubSDhDAAAGM2gkPag52zsWtalrSGcAQAAGzNuSf6z/3rtPiHte6r1VlVhZFW1kGRpaWkpCwsLk+4OAABM1kYLiKxnBtelLS8vZ3FxMUkWW2vL417HyBkAALDvRhlNu/kdh19njtelCWcAAMDmGRTSnnD6xq41Z1MehTMAAGDz9QtpNUL8+Mw71+6bk5DmOWcAAMDW2ejz0v7pGVvbnymmIMiYFAQBAIAxDSoectNDkysvHXz+b30+OflOnddTUEBEQRAAAGA2DVqX9pQPbX9/poRwBgAATEa/kLbfDYef96l/3Lo+TZA1ZwAAwGRtdF3a6b+993W7bmv6NAFGzgAAgOkxyvPSbnyznl9qy7u0XYQzAABg+gwKaU/uWZdWwhkAAMDW6xfSdt1kcv3ZQtacAQAA02+j69JmkHAGAADMlt6gtoOY1ggAADAFhDMAAIApIJwBAABMAeEMAABgCghnAAAAU0A4AwAAmALCGQAAwBQQzgAAAKaAcAYAADAFhDMAAIApIJwBAABMAeEMAABgCghnAAAAU0A4AwAAmALCGQAAwBQQzgAAAKaAcAYAADAFhDMAAIApIJwBAABMAeEMAABgCkxFOKuqX62qL1bV1VV1TlXdd8TzjquqVlVvW7X/1O7+3u2jq9ocUFUvrarLqurKqnpHVd1mMz8XAADAqCYezqrq0UlekuR5Se6Z5MNJTq+qw4ecd7skJ3fb9/MvSW7Vsx2z6vhLkjwiyXFJ7pPkwCTvqqr9xvskAAAA45t4OEvyzCSntNZe01r7TGvt6Um+kuSp653QDVCvT/LsJF9Yp9nu1trFPdvlPecvJnlSkt9srb23tfbxJL+U5O5JHrQ5HwsAAGB0Ew1nVbUryVFJzlh16Iwk9x5w6olJvt5aO2VAmwdU1aVV9bmqenVVHdpz7KgkN+x939bahUnOW+99u9MgF1a2JAcNeG8AAIANmfTI2S2S7JfkklX7L0lyWL8Tquon0hn1evKA656e5LFJfirJbya5V5L3V9UB3eOHJdnTWvvmqO+b5IQkSz3bVwe8PwAAwIbsP+kOdLVVv1effamqg5K8LsmTW2uXrXux1t7c8+t5VXV2kguS/EyStw7oR9/37XpBkhf3/H5QBDQAAGCTTDqcXZbk2qwdrTo0a0fTkuSOSW6f5J1VtbLvBklSVd9NcufW2n+vPqm1dlFVXZDkiO6ui5PsqqqbrRo9OzTJR/p1tLW2O8nuld973h8AAGCfTXRaY2ttT5Jzkhy96tDR6R+SPptO0Y4je7Z3JPlA9/VX+r1PVR2S5LZJLuruOifJNb3vW1W3SnK3dd4XAABgS0165CzpTBU8rTv18KwkT0lyeJJXJklVvTbJ11prJ7TWrk6naMf3VNW3kqS1dl739wOTnJTkH9IJY7dP8vx0Run+sdt2qapOSfKiqvpGksvTKcv/ySTv3cLPCgAA0NfEw1lr7c3dka0T03ke2XlJjmmtXdBtcniS6zZwyWvTGV17XJKD0wloH0jy6NbaFT3tnpHku0nekuTGSd6X5PjW2rX78HEAAADGUq2tV/+CQbrl9JeWlpaysLAw6e4AAAATsry8nMXFxSRZbK0tj3udSZfSBwAAIMIZAADAVJj4mrNZt7w89qglAACwA2xWJrDmbExV9f3xEGoAAGCv27TWvjbuycLZmKrzFOpbJ7liWFs2xUHphOHbxP/ms8j9m33u4exzD2efezj73MPZN+geHpTkwrYPAcu0xjF1/0cfOxWzMZ0snCS5Yl8q4DAZ7t/scw9nn3s4+9zD2ecezr4h93Cf76mCIAAAAFNAOAMAAJgCwhmzYneS53R/Mnvcv9nnHs4+93D2uYezzz2cfVt6DxUEAQAAmAJGzgAAAKaAcAYAADAFhDMAAIApIJwBAABMAeGMqVFVJ1VVW7Vd3HO8um0urKrvVNUHq+quk+zzvKuq+1XVO7v3pFXVw1cdH3rPqupmVXVaVS11t9Oq6uDt/STza4R7eGqf7+VHV7U5oKpeWlWXVdWVVfWOqrrN9n6S+VRVJ1TVx6rqiqq6tKreVlV3XtVm6P2pqsO7/z+4stvuL6pq1/Z+mvk04j38YJ/v4ZtWtfFn6YRU1VOr6j+rarm7nVVVP91z3Hdwyo1wD7ftOyicMW0+leRWPdvde479TpJnJvm1JPdKcnGS91TVQdvdSb7npkk+kc496WeUe/aGJEcmeWh3OzLJaVvVYdYYdg+T5F9y/e/lMauOvyTJI5Icl+Q+SQ5M8q6q2m/Te8tq90/y8iQ/luToJPsnOaOqbtrTZuD96f78p3T+v3CfbrtHJXnRNn2GeTfKPUySV+f638NfWXXcn6WT89Ukv5fkh7vb+5O8vecfI30Hp9+we5hs13ewtWazTcWW5KQk/7HOsUpyUZLf7dl3QJJvJfmVSffd1pKkJXn4Ru5Zkh/snvejPW1+rLvvzpP+TPO2rb6H3X2nJnnbgHMWk+xJ8uiefbdOcm2Sh0z6M83bluT7uvfxfqPenyQ/3f391j1tjktydZKFSX+medtW38Puvg8mecmAc/xZOmVbksuTPMl3cHa3lXvYfb1t30EjZ0ybI7rTq75YVW+qqv+vu/8OSQ5LcsZKw9ba7iQfSnLvCfST4Ua5Zz+eZKm19m89bT6aZCnu6zR5QHe61eeq6tVVdWjPsaOS3DDXv88XJjkv7uEkLHZ/Xt79Ocr9+fEk53X3r3h3Ov+YctSW9pZ+Vt/DFY/tTnf7VFWdvGoGgj9Lp0RV7VdVx6UzCnZWfAdnTp97uGJbvoP770PfYbP9W5LHJflcklsm+YMkH+kOKR/WbXPJqnMuSXK7beshGzHKPTssyaV9zr2053wm6/Qkf5fkgnQC93OTvL+qjuqG7cOS7GmtfXPVeZfEPdxWVVVJXpzkX1tr53V3j3J/Dsuq72lr7ZtVtSfu4bZa5x4myeuTfDGdqeF3S/KCJPdIZxpk4s/Siauqu6fzF/kbJfl2kke01j5dVUfGd3AmrHcPu4e37TsonDE1Wmun9/z6yao6K8l/J3l8kpUCBG3VadVnH9Nl2D3rd//c1ynRWntzz6/nVdXZ6QS1n0ny1gGnuofb72VJ/mc6a1aG8T2cTn3vYWvt1T2/nldV5yc5u6p+qLV27kqzPtdzD7fPf6WzxujgdNaL/W1V3X9Ae9/B6dP3HrbWPr2d30HTGplarbUrk3wyyRHp/EtFsvZfHw7N2pEZpsMo9+zidEZJV/u+uK9TqbV2UTrh7IjurouT7Kqqm61q6ru5jarqpUkeluQnW2tf7Tk0yv25OKu+p932N4x7uG0G3MN+zk1yTa7/PfRn6QS11va01j7fWju7tXZCOoWW/nd8B2fGgHvYz5Z9B4UzplZVHZDOAsuLsnco+eie47vSqXL1kYl0kGFGuWdnJVmsqh/pafOj6ay5cF+nUFUdkuS26Xwvk+ScdP4D1Xufb5XOtA/3cItVx8uSPDLJT7XWvriqySj356wkd+vuX/HgJLu757OFRriH/dw1nb+4r3wP/Vk6fSqdNWO+g7Nr5R72s2XfwepWE4GJq6qTk7wzyZfT+RelP0jnL/J3b61dUFW/m+SEJE9Icn6SZyV5QDpVcK6YSKfnXFUdmORO3V8/nk7Z/A8kuby19uVR7llVnZ5O5aqVkrSvSnJBa+3ntutzzLNB97C7nZTkH9L5D9Dtkzw/yeFJfrDnHv5lkp9Ncnz3nJOTHJLkqNbatdvzSeZTVb0iyS8mOTadKTkrllpr3+m2GXh/umW8/yOdf9397SQ3z94qnb++PZ9kfg27h1V1xySPTfLPSS5Lcpd0Sqx/J8m9Vr5j/iydnKp6fjrrc7+S5KB0Ki3+XpKHttbe4zs4/QbdwyRfyHZ+ByddptJmW9mSvCnJhemUnP1aOn8hvEvP8UrnL4oXpVNe9kNJ7jbpfs/zlk7Qan22U0e9Z+n8R+h1SZa72+uSHDzpzzYv26B7mOTG6VQMu7T7vbygu/+2q65xoyQvTfKNJFel848st53E55m3bZ1715Icv5H7k07gflf3+De67Q+Y9Oebh23YPUxnpPpD3fuyO8nnk/x5kpuvuo4/Syd3D09J8qXu/bk0yXuTHN1z3HdwyrdB93C7v4NGzgAAAKaANWcAAABTQDgDAACYAsIZAADAFBDOAAAApoBwBgAAMAWEMwAAgCkgnAEAAEwB4QwAAGAKCGcAMERVfamqnj7pfgCwswlnANBVVcdX1bf6HLpXkldtw/sLgQBzbP9JdwAApl1r7euT7sNGVNWu1tqeSfcDgI0xcgbA1KmqD1bVX1TVC6vq8qq6uKpOGvHcxap6VVVdWlXLVfX+qrpHz/F7VNUHquqK7vFzquqHq+oBSf4myWJVte52Uvec641odY/9SlW9q6quqqrPVNWPV9Wdun2/sqrOqqo79pxzx6p6e1VdUlXfrqqPVdWDej9zktsl+bOV9+859qiq+lRV7e725TdXfeYvVdUfVNWpVbWU5NVVtauqXlZVF1XV1d02J2zoRgCwrYQzAKbV45NcmeRHk/xOkhOr6uhBJ1RVJfmnJIclOSbJUUnOTfK+qrp5t9nrk3w1namKRyX5kyTXJPlIkqcnWU5yq+528oC3+8Mkr01yZJLPJnlDkr9K8oIkP9xt87Ke9gcm+eckD0pyzyTvTvLOqjq8e/yR3X6d2PP+qaqjkrwlyZuS3D3JSUmeW1XHr+rPbyc5r/uZnpvkN5I8LMkvJLlzkl9K8qUBnweACTOtEYBp9Z+tted0X59fVb+W5IFJ3jPgnJ9MJ8Ac2lrb3d33W1X18CT/K511Y4cn+dPW2mdXrr1ycnfUqbXWLh6hf3/TWntL97z/k+SsJM9trb27u+/P0xmJSzoX/USST/Sc/wdV9Yh0AtTLWmuXV9W1Sa5Y9f7PTPK+1tpzu79/rqrukk4YO7Wn3ftba98Lk93Qd36Sf22ttSQXjPCZAJggI2cATKv/XPX7RUkOHXLOUemMUH2jO3Xw21X17SR3SLIyxfDFSV5TVe+tqt/rnXq4D/27pPvzk6v23aiqFpKkqm7anab56ar6VrdfP5BOWBzkB5OcuWrfmUmOqKr9evadvarNqemM6v1Xd4rog4d+IgAmSjgDYFpds+r3luH/3bpBOiHuyFXbnZP8aZK01k5Kctd0pj/+VJJPd0ew9qV/bcC+lT7/aZJHJfn9JPft9uuTSXYNeZ/quVbvvtWu7P2ltXZuOqH0D5PcOMlbqurvh7wXABNkWiMAO8m56aw3+25r7UvrNWqtfS7J59IpvvHGJE9I8o9J9iTZb73z9tF9k5zaWvvHJKmqA5PcflWbfu//6ST3WbXv3kk+11q7dtAbttaWk7w5yZu7wexfqurmrbXLx/sIAGwlI2cA7CTvTWft19uq6iFVdfuqundV/XG3IuONuxUMH1BVt6uqn0inMMhnuud/KcmBVfXAqrpFVd1kE/v2+SSPrKoju9Uj35C1/x3+UpL7VdX3V9UtuvtelOSBVfWHVfU/qurxSX4tg4uVpKqeUVXHVdUPVNX/SPLzSS5O0u85bgBMAeEMgB2jW/jimCT/N8lfpzM69qZ0RqguSXJtkkPSqbL4uXSqIJ6e5Nnd8z+S5JXpjDZ9PZ0qkZvlGUm+mU5VyHemU63x3FVtTuz29b+7778yPfEXkhyXTjXGP0pyYmvt1CHv9+0kv5vOWrSPda97TGvtun3+JABsier8dwwAAIBJMnIGAAAwBYQzAGZGVT22t0T+qu1Tk+4fAOwL0xoBmBlVdVCSW65z+JrWmgctAzCzhDMAAIApYFojAADAFBDOAAAApoBwBgAAMAWEMwAAgCkgnAEAAEwB4QwAAGAKCGcAAABT4P8BABQEE7L61QwAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 1000x1000 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "cvresult = pd.DataFrame.from_csv('1_nestimators.csv')\n",
    "\n",
    "cvresult = cvresult.iloc[25:]\n",
    "# plot\n",
    "test_means = cvresult['test-mlogloss-mean']\n",
    "test_stds = cvresult['test-mlogloss-std'] \n",
    "        \n",
    "train_means = cvresult['train-mlogloss-mean']\n",
    "train_stds = cvresult['train-mlogloss-std'] \n",
    "\n",
    "x_axis = range(25,cvresult.shape[0]+25)\n",
    "        \n",
    "fig = pyplot.figure(figsize=(10, 10), dpi=100)\n",
    "pyplot.errorbar(x_axis, test_means, yerr=test_stds ,label='Test')\n",
    "pyplot.errorbar(x_axis, train_means, yerr=train_stds ,label='Train')\n",
    "pyplot.title(\"XGBoost n_estimators vs Log Loss\")\n",
    "pyplot.xlabel( 'n_estimators' )\n",
    "pyplot.ylabel( 'Log Loss' )\n",
    "pyplot.savefig( 'n_estimators_detail.png' )\n",
    "\n",
    "pyplot.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>n_estimators</th>\n",
       "      <th>test-mlogloss-mean</th>\n",
       "      <th>test-mlogloss-std</th>\n",
       "      <th>train-mlogloss-mean</th>\n",
       "      <th>train-mlogloss-std</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>333</th>\n",
       "      <td>333</td>\n",
       "      <td>0.586414</td>\n",
       "      <td>0.001854</td>\n",
       "      <td>0.459538</td>\n",
       "      <td>0.001172</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>334</th>\n",
       "      <td>334</td>\n",
       "      <td>0.586378</td>\n",
       "      <td>0.001823</td>\n",
       "      <td>0.459270</td>\n",
       "      <td>0.001183</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>335</th>\n",
       "      <td>335</td>\n",
       "      <td>0.586423</td>\n",
       "      <td>0.001817</td>\n",
       "      <td>0.458977</td>\n",
       "      <td>0.001130</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>336</th>\n",
       "      <td>336</td>\n",
       "      <td>0.586373</td>\n",
       "      <td>0.001819</td>\n",
       "      <td>0.458641</td>\n",
       "      <td>0.001121</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>337</th>\n",
       "      <td>337</td>\n",
       "      <td>0.586325</td>\n",
       "      <td>0.001805</td>\n",
       "      <td>0.458323</td>\n",
       "      <td>0.001151</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     n_estimators  test-mlogloss-mean  test-mlogloss-std  train-mlogloss-mean  \\\n",
       "333           333            0.586414           0.001854             0.459538   \n",
       "334           334            0.586378           0.001823             0.459270   \n",
       "335           335            0.586423           0.001817             0.458977   \n",
       "336           336            0.586373           0.001819             0.458641   \n",
       "337           337            0.586325           0.001805             0.458323   \n",
       "\n",
       "     train-mlogloss-std  \n",
       "333            0.001172  \n",
       "334            0.001183  \n",
       "335            0.001130  \n",
       "336            0.001121  \n",
       "337            0.001151  "
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_nestimators = pd.read_csv(\"1_nestimators.csv\")\n",
    "data_nestimators.tail()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "338"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "xgb1.get_params()['n_estimators']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "因为在表格里面的n_estimators是337，我以为最佳的弱学习器数目就是337，后来发现他是从0开始编号的，同学提醒了一下，发现确实弱学习器的最佳数量是338  \n",
    "\n",
    "由上表可知，其他值都是缺省值的条件下，最佳的n_estimators值为338，此时在最优结果logloss = 0.586325"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
